Commit Graph

714 Commits (8d410bdfcb2905cecd99fcf6ff67cd829aeb9da9)

Author SHA1 Message Date
Xiang Li 866a9d4e41 Merge pull request #2568 from xiang90/raftnode
raft: make node configurable
2015-03-24 11:18:22 -07:00
Yicheng Qin ea78f5d1aa Merge pull request #2552 from yichengq/fix-2396
etcdserver: check -initial-cluster in join case
2015-03-23 22:46:38 -07:00
Yicheng Qin abcd828114 etcdserver: add join-existing check 2015-03-23 22:31:20 -07:00
Xiang Li abddef0f28 raft: make node configurable 2015-03-23 21:20:49 -07:00
Kelsey Hightower 4611c3b2d7 netutil: add BasicAuth function
etcd ships it's own BasicAuth function and no longer requires
Go 1.4 to build.
2015-03-20 17:32:33 -07:00
Xiang Li 9d28f94005 etcdserver/etcdhttp: do not return back the password of a user 2015-03-16 22:35:01 -07:00
Xiang Li f3e4dbf967 etcdserver/etcdhttp: write the http error to response writer 2015-03-16 15:24:19 -07:00
Xiang Li bba7f75562 Merge pull request #2517 from yichengq/fix-sec2
security: fix var shadowing in CreateOrUpdateUser
2015-03-16 15:08:55 -07:00
Yicheng Qin 8335a5407b security: fix var shadowing in CreateOrUpdateUser 2015-03-16 14:59:05 -07:00
Yicheng Qin d7780cf293 security: fix var shadowing in CreateOrUpdate 2015-03-16 14:55:04 -07:00
Barak Michener 001efa0639 security: Implement RBAC security for etcd
stub out security

further wip

Last stub before CRUD for roles

Complete role merging

start tests

add Godep for golang.org/x/crypto/bcrypt

first round of comments

add tests, remove root addition (will be added back as part of creation)

Add security checks for /v2/machines and /v2/keys

Allow non-root to determine if security is enabled, get machine list.

Responding to comments, remove multiple verbs (like /v2/security/user/foo/password)

add some prefixes to the logging
2015-03-16 16:23:11 -04:00
Xiang Li d015610da5 etcdserver: separate apply and raft routine 2015-03-10 13:34:24 -07:00
Yicheng Qin b4b9b9118a rafthttp: report MsgSnap status 2015-03-02 09:38:11 -08:00
Yicheng Qin 9989bf1d36 Merge pull request #2407 from yichengq/334
rafthttp: report unreachable status of the peer
2015-03-02 09:35:35 -08:00
Yicheng Qin 9b986fb4c1 rafthttp: report unreachable status of the peer
When it failed to send message to the remote peer, it reports unreachable
to raft.
2015-03-01 16:48:26 -08:00
Xiang Li 428b77afc3 etcdserver: keep a min number of entries in memory
Do not aggressively compact raft log entries. After a snapshot,
etcd server can compact the raft log upto snapshot index. etcd server
compacts to an index smaller than snapshot to keep some entries in memory.
The leader can still read out the in memory entries to send to a slightly
slow follower. If all the entries are compacted, the leader will send the
whole snapshot or read entries from disk if possible.
2015-03-01 10:12:13 -08:00
Xiang Li a4dab7ad75 *: do not block etcdserver when encoding store into json
Encoding store into json snapshot has quite high CPU cost. And it
will block for a while. This commit makes the encoding process non-
blocking by running it in another go-routine.
2015-02-28 11:41:58 -08:00
Xiang Li 9b4d52ee73 raft: do not resend snapshot if not necessary
raft relies on the link layer to report the status of the sent snapshot.
If the snapshot is still sending, the replication to that remote peer will
be paused. If the snapshot finish sending, the replication will begin
optimistically after electionTimeout. If the snapshot fails, raft will
try to resend it.
2015-02-28 11:41:58 -08:00
Xiang Li 86429264fb wal: support auto-cut in wal
WAL should control the cut logic itself. We want to do falloc to
per allocate the space for a segmented wal file at the beginning
and cut it when it size reaches the limit.
2015-02-28 11:18:59 -08:00
Xiang Li 95bba154d6 etcdserver: add propose summary 2015-02-28 11:16:42 -08:00
Xiang Li 83c953b153 etcdhttp: move /stats to /debug/vars 2015-02-28 11:16:42 -08:00
Xiang Li 84485643fe *: expose wal metrics at /metrics 2015-02-28 11:06:11 -08:00
Xiang Li 2af33fd494 raft: add reportUnreachable 2015-02-28 10:45:22 -08:00
Brian Waldon 4a77760f56 client: break dependency on httptypes pkg 2015-02-28 10:38:46 -08:00
Xiang Li 2e078582f9 etcdmain: expose runtime metrics 2015-02-28 10:11:53 -08:00
Xiang Li 33afbfead6 etcdserver: remove the dep on metrics. first step towards removing metrics pkg from etcd. 2015-02-28 10:09:55 -08:00
Xiang Li 5ede18be74 raft: separate compact and createsnap in memory storage 2015-02-28 10:08:30 -08:00
Yicheng Qin cff005777a etcdserver: fix cluster fallback recovery
Cluster and transport may recover to old states when new node joins
the cluster. Record cluster last modified index to avoid this.
2015-02-20 14:30:00 -08:00
Barak Michener 92dca0af0f *: remove shadowing of variables from etcd and add travis test
We've been bitten by this enough times that I wrote a tool so that
it never happens again.
2015-02-17 16:31:42 -05:00
Xiang Li beb44ef6ba etcdserver: fix error message when valide the discovery cluster 2015-02-16 09:53:01 -08:00
Xiang Li 73e67628d9 Merge pull request #2313 from xiang90/cluster_mu
etcdserver: move the mutex before what it guards
2015-02-14 23:05:53 -08:00
Xiang Li 04bd06d20b etcdserver: move the mutex before what it guards 2015-02-14 22:26:12 -08:00
Xiang Li c5ca1218f3 etcdserver: GetClusterFromPeers -> GetClusterFromRemotePeers 2015-02-13 19:05:29 -08:00
Xiang Li f7540912d6 etcdserver: getOtherPeerURLs -> getRemotePeerURLs 2015-02-13 18:56:45 -08:00
Xiang Li cfa7ab6074 etcdserver: validate discovery cluster 2015-02-13 14:32:24 -08:00
Xiang Li c16cc3a6a3 etcdserver: recover transport when recovering from a snapshot 2015-02-13 10:16:28 -08:00
Xiang Li fbc4c8efb5 etcdserver: fix snapshot 2015-02-13 09:54:25 -08:00
Barak Michener a0e3bc9cbd etcdserver: Unmask the snapshotter. Fixes #2295 2015-02-13 11:56:00 -05:00
Barak Michener cd50f0e058 etcdserver: Create MemberDir() and base {Snap,WAL}Dir() thereon. Audit DataDir. 2015-02-12 12:45:19 -05:00
Barak Michener fade9b6065 etcdserver: Refactor 2.0.1 directory rename into a proper migration
fix all instances

fix detection test
2015-02-12 11:53:19 -05:00
Xiang Li 163f0f09f6 etcdserver: cleanup cluster_util 2015-02-11 16:20:38 -08:00
Xiang Li 20497f1f85 etcdserver: move remote cluster retrive to cluster_util.go 2015-02-11 14:03:14 -08:00
Xiang Li 6e1aecfc6f etcdserver: save confstate when apply new snapshot 2015-02-10 07:31:25 -08:00
Yicheng Qin f13c7872d5 etcdserver: register pre-defined namespaces in store 2015-02-04 16:33:40 -08:00
Yicheng Qin 7840d49ae0 etcdserver: not add self to transporter based on local ID
If this is decided by local name, it comes to trouble if the name is
duplicate in the cluster.
2015-01-29 12:35:47 -08:00
Xiang Li 276c9540b4 etcdserver: support raft.status 2015-01-26 16:39:33 -08:00
Yicheng Qin f0c9a54edb Merge pull request #2156 from yichengq/309
pkg/metrics: self-manage global expvar map
2015-01-26 16:20:31 -08:00
Yicheng Qin 08b34a3f5b pkg/metrics: self-manage global expvar map
This helps the embedded tests.
2015-01-26 16:20:09 -08:00
Shota Fukumori (sora_h) f8ce5996b0 Treat URLs have same IP address as same
- To solve validation error problem using URLs in hostname #2123
2015-01-27 04:36:41 +09:00
Xiang Li 9c7f66c5d9 Merge pull request #2119 from sorah/peer-ca-on-fetching-members
etcdserver: User peerTLSInfo to get cluster member
2015-01-26 10:50:44 -08:00
Shota Fukumori (sora_h) 033e7d1db9 etcdserver: User peerTLSInfo to get cluster member 2015-01-27 03:43:21 +09:00
Jonathan Boulle f1ed69e883 *: switch to line comments for copyright
Build tags are not compatible with block comments.
Also adds copyright header to a few places it was missing.
2015-01-26 09:53:30 -08:00
Xiang Li a77bf97c14 etcdhttp: add internalVersion 2015-01-22 15:42:16 -08:00
Ben Darnell 59214978a2 raft: Add applied index as an argument to newRaft and RestartNode. 2015-01-22 11:38:05 -05:00
Yicheng Qin 99821579bf metrics: add /rafthttp/stream metrics 2015-01-21 13:24:21 -08:00
Xiang Li 0eaaad0e48 raft: add Status interface
Status returns the current status of raft state machine.
2015-01-16 14:02:04 -08:00
Xiang Li a97f331a0e etcdhttp: add health endpoint 2015-01-16 10:52:02 -08:00
Xiang Li 973f79e1c9 etcdserver: separate out raft related stuff 2015-01-15 15:15:13 -08:00
Xiang Li 1a6161d08a Merge pull request #2104 from xiang90/timeout
etcdserver: make heartbeat/election configurable
2015-01-15 13:52:20 -08:00
Xiang Li 276a4abac0 etcdserver: make heartbeat/election configurable 2015-01-15 11:11:33 -08:00
Yicheng Qin 190fd446f9 pkg/types: add URLs tests 2015-01-15 10:24:23 -08:00
Xiang Li 89d95539cf Merge pull request #2083 from yichengq/293
*: move etcdserver/idutil -> pkg/idutil
2015-01-13 13:04:50 -08:00
Yicheng Qin 07a69430c1 *: move etcdserver/idutil -> pkg/idutil 2015-01-13 11:54:51 -08:00
Yicheng Qin dc6aef0d02 etcdhttp: add NewPeerHandler test 2015-01-12 15:56:29 -08:00
Yicheng Qin 270e67db84 wal: not export unnecessary public functions 2015-01-09 14:55:10 -08:00
Yicheng Qin bca1e5aea6 Merge pull request #2057 from yichengq/282
fix context time-out failure on travis
2015-01-07 13:41:26 -08:00
Yicheng Qin 930156c18a integration: adjust election ticks using env var 2015-01-07 11:18:29 -08:00
Yicheng Qin 6b237416e1 Merge pull request #2044 from yichengq/278
wal: record mark when snapshotting
2015-01-07 08:26:33 -08:00
Yicheng Qin 6460e49a33 wal: save empty snapshot when create
So caller can open at empty snapshot to read all entries.
2015-01-06 19:48:21 -08:00
Yicheng Qin 84f62f21ee wal: record and check snapshot 2015-01-06 16:27:40 -08:00
Xiang Li 1ebad5e42c etcdhttp: support member/leader endpoint 2015-01-06 08:52:33 -08:00
Xiang Li 6b8667152b Merge pull request #2035 from xiang90/errorc
etcdserver: collect error from errorc
2015-01-05 11:29:01 -08:00
Yicheng Qin cb5bff5b05 Merge pull request #2034 from yichengq/276
etcdhttp: reset serve and watch timeout
2015-01-05 08:33:25 -08:00
Xiang Li 15be030aaa etcdserver: collect error from errorc 2015-01-02 20:13:46 -08:00
Yicheng Qin 4dd00be365 etcdhttp: reset serve and watch timeout 2015-01-02 16:39:13 -08:00
Xiang Li 2a83e350b1 Merge pull request #1992 from xiang90/rm_leader
*: support removing the leader from a 2 members cluster
2015-01-02 14:15:12 -08:00
Xiang Li 6e727625b9 etcdserver: continue to apply after self-removed 2015-01-02 14:10:07 -08:00
Xiang Li 51ffc88096 etcdsever: remove mult_server_test 2015-01-02 13:49:58 -08:00
Xiang Li 41f6137261 etcdserver: use the actual store implementation when we need the actual implementation 2015-01-02 12:22:01 -08:00
Xiang Li 27d47977d9 etcdserver: move recorder to testutil 2015-01-02 11:21:23 -08:00
Xiang Li ac6cd03365 etcdserver: refactor server_test.go 2015-01-02 10:56:09 -08:00
Xiang Li 04003a01ba Merge pull request #2013 from xiang90/tr
rafthttp cleanup
2014-12-31 08:35:20 -08:00
Xiang Li 803c38f448 etcdserver: move error to errors.go
Both server.go and cluster.go are using defined ErrX. Move error
to errors.go
2014-12-30 15:02:07 -08:00
Xiang Li c3d2f5eea0 pbutil: add getbool to pbutil 2014-12-30 14:51:26 -08:00
Yicheng Qin 241a474935 etcdserver: refactor server tests
1. remove redundant fake struct
2. use fake node for better testing
3. code clean
2014-12-30 13:49:55 -08:00
Yicheng Qin a92bd1d165 etcdserver: add multi_server_test.go 2014-12-30 00:10:18 -08:00
Xiang Li f79b9042ab etcdserver: fix streaming handler 2014-12-29 22:12:58 -08:00
Yicheng Qin 3748088b96 etcdserver: print out log of normal tests 2014-12-29 14:38:00 -08:00
Yicheng Qin 6ccaadc95d Merge pull request #1952 from yichengq/262
etcdserver: add id generator
2014-12-29 13:59:06 -08:00
Yicheng Qin 05c921229e etcdserver: add id generator 2014-12-29 13:03:04 -08:00
Xiang Li c712dd682a rafthttp: make Transport private 2014-12-29 12:20:52 -08:00
Xiang Li a14d13f724 rafthttp: make fields in Transport private 2014-12-29 12:08:13 -08:00
Xiang Li 7c8b9c0203 Merge pull request #2011 from xiang90/timeutil
etcdserver: move getExpr to timeutil
2014-12-29 12:03:25 -08:00
Xiang Li 152676f43a *: support removing the leader from a 2 members cluster 2014-12-29 11:34:33 -08:00
Yicheng Qin 5bb8eeb5cf rafthttp: transport cleanup 2014-12-29 11:21:40 -08:00
Xiang Li cea29fe158 etcdserver: move getExpr to timeutil 2014-12-29 11:15:02 -08:00
Yicheng Qin 08f839e32c rafthttp: set the API boundary of the package 2014-12-28 15:50:27 -08:00
Xiang Li 1535596252 etcdserver: remove unnecessary indirection 2014-12-26 11:03:13 -08:00
Xiang Li 3dcd66459d etcdserver: remove unused containsUint64() 2014-12-26 10:56:56 -08:00
Xiang Li 69444b6bba etcdserver: cleanup server.go 2014-12-25 21:37:20 -08:00
Xiang Li 78b51d3f2f etcdserver: cleanup cluster.go 2014-12-25 20:56:30 -08:00
Xiang Li 6dc3af5da4 etcdserver: cluster clean up 2014-12-25 20:36:48 -08:00
Xiang Li 7a5bf53222 etcdserver: move member sort interface to member.go 2014-12-25 20:18:55 -08:00
Xiang Li ef0a66bb0a etcdserver: kill a todo in test 2014-12-25 18:14:05 -08:00
Xiang Li f43bc809b9 etcdserver: cleanup wal upgrade 2014-12-24 22:02:46 -08:00
Xiang Li b9d228b0fa etcdserver: remove example.go 2014-12-24 21:41:31 -08:00
Jonathan Boulle 4e6cbc937e *: change remaining 0.5 references -> 2.0 2014-12-18 16:14:42 -08:00
Xiang Li c27c288bef etcdserver: update stats when become leader 2014-12-15 17:02:48 -08:00
Xiang Li 04522baeee etcdserver: fix leader stats 2014-12-15 16:50:03 -08:00
Xiang Li 53bf7e4b5e wal: rename openAtIndex -> open; OpenAtIndexUntilUsing -> openNotInUse 2014-12-14 19:33:06 -08:00
Xiang Li ea94d19147 *: lock the in using files; do not purge locked the wal files 2014-12-14 19:27:22 -08:00
Yicheng Qin dcf34c0ab4 Merge pull request #1938 from yichengq/262
etcdserver: protect the sender map in SendHub
2014-12-15 10:41:52 +08:00
Yicheng Qin ceb077424d etcdserver: protect the sender map in SendHub 2014-12-15 10:37:41 +08:00
Xiang Li d86603840d rafthttp: better logging 2014-12-14 09:50:59 -08:00
Xiang Li 4724cbbe2c etcdserver: one line 2014-12-11 22:17:36 -08:00
Xiang Li 935f7128a9 etcdserver: move stats inferface to stats pkg 2014-12-11 22:14:05 -08:00
Barak Michener 5f16fab541 Merge pull request #1915 from barakmich/1834
Return Unknown instead of NotExist
2014-12-11 13:49:26 -05:00
Barak Michener cf7690cb51 detect more cases of empty directories and actual errors 2014-12-11 13:37:32 -05:00
Xiang Li c26542b7f2 Merge pull request #1913 from xiang90/lazy_snap_dir
etcdserver: create snap dir until start the node
2014-12-11 09:39:51 -08:00
Xiang Li 836ccabad2 etcdserver: create snap dir until start the node 2014-12-11 09:25:18 -08:00
Yicheng Qin 4777cba995 Merge pull request #1898 from robszumski/improve-logging
Improve logging for etcdserver and rafthttp
2014-12-10 10:46:47 -08:00
Nikhil Sarda a852936a59 etcdserver: removed an unhelpful test failure message
this commit changes instances of "blah" in a test to more
descriptive messages
2014-12-09 21:45:50 -08:00
Rob Szumski 13f3158728 etcdserver: improve discovery ignore warning 2014-12-09 15:57:25 -08:00
Xiang Li e4c0f5c1a8 Merge pull request #1895 from xiang90/snap_nodes
etcd: update conf when apply the confChange entry
2014-12-09 11:45:01 -08:00
Xiang Li a5efbf826d raft: drop nodes in softState 2014-12-09 11:43:52 -08:00
Xiang Li 29d7a2a558 etcd: update conf when apply the confChange entry 2014-12-08 23:37:07 -08:00
Yicheng Qin 4804c45e14 raft: set raft.Commit too when setting raftLog.committed 2014-12-08 22:35:55 -08:00
Yicheng Qin 9c8f5c9535 Merge pull request #1891 from yichengq/257
etcdserver: init state before run loop correctly
2014-12-08 16:38:33 -08:00
Yicheng Qin 13814c9d7d etcdserver: init state before run loop correctly 2014-12-08 16:13:16 -08:00
Yicheng Qin 7e06d85651 etcdserver: apply entries when it is not empty
Or it updates appliedi wrongly.
2014-12-08 15:56:38 -08:00
Yicheng Qin 71f3b80fbe etcdserver: check recovery error when new server 2014-12-08 14:55:23 -08:00
Yicheng Qin 8c338ffcc7 etcdserver: correct the log about recovering from snapshot 2014-12-08 14:51:42 -08:00
Yicheng Qin 771ff4589d etcdserver: not add self into sendhub when new server 2014-12-05 00:18:40 -08:00
Yicheng Qin 1d1c2ff834 Merge pull request #1841 from yichengq/246
etcdserver: close storage when stop
2014-12-04 15:36:24 -08:00
Yicheng Qin a7bc03b42b etcdserver: close storage when stop 2014-12-04 15:16:22 -08:00
Xiang Li 88e2fab572 Merge pull request #1859 from xiang90/pause_test
*: add pauseMember test
2014-12-04 15:11:59 -08:00
Veres Lajos 3de2ab2c04 *: typofixes
https://github.com/vlajos/misspell_fixer
2014-12-04 22:51:19 +00:00
Xiang Li 151f043414 *: add pauseMember test 2014-12-04 14:22:43 -08:00
Yicheng Qin fa292391d8 etcdserver: close idle connections when stop sendhub 2014-12-02 00:08:47 -08:00
Xiang Li 7beac083ff Merge pull request #1810 from xiang90/purge
*: support purging old wal/snap files
2014-12-01 12:05:05 -08:00
Xiang Li d3db010190 *: support purging old wal/snap files 2014-12-01 11:50:17 -08:00
Xiang Li bc5acd3c42 etcdserver: log snapshot event 2014-11-30 12:10:20 -08:00
Xiang Li e23f9e76d1 raft: do not applysnapshot in raft 2014-11-26 10:59:13 -08:00
Xiang Li 9df0e7715d raft: do not panic on out of date compaction 2014-11-25 15:14:39 -08:00
Xiang Li 01cbcce8ba etcdserver: do not applySnapshot twice 2014-11-25 14:53:49 -08:00
Xiang Li 74d8c7f457 etcdserver: cleanup main loop 2014-11-25 14:38:18 -08:00
Yicheng Qin 7e6e305c4f Merge branch 'log_interface'
Conflicts:
	raft/raft.go
2014-11-25 14:22:11 -08:00
Yicheng Qin a13d5a70ff etcdserver: save snapshot before entries 2014-11-25 12:39:15 -08:00
Owen Smith c67b937d62 etcdserver: truncate WAL from correct index when forcing new cluster
When loading from a backup with a snapshot and WAL, the length of WAL entries
can be lower than the current index integer value, causing a panic when
slicing off uncommitted entries. This looks for WAL entries higher than
the current index before slicing.
2014-11-25 16:46:56 +00:00
Yicheng Qin 54e1237271 etcdserver: panic when snapshot on raft storage
Snapshot on raft storage should always succeed. If there is an error, it must
be internal fault and needs stack info to debug.
2014-11-24 21:22:49 -08:00