Commit Graph

307 Commits (9f8342dba43dbd5211656416f6c2291410a1378b)

Author SHA1 Message Date
Yicheng Qin e77f8e311c etcdserver: parse context error for better message 2014-10-27 10:59:15 -07:00
Xiang Li 009b737cef *: better logging 2014-10-26 08:13:03 -07:00
Jonathan Boulle 719c57a29d proxy: retrieve ClientURLs from cluster
This is a simple solution to having the proxy keep up to date with the
state of the cluster. Basically, it uses the cluster configuration
provided at start up (i.e. with `-initial-cluster-state`) to determine
where to reach peer(s) in the cluster, and then it will periodically hit
the `/members` endpoint of those peer(s) (using the same mechanism that
`-cluster-state=existing` does to initialise) to update the set of valid
client URLs to proxy to.

This does not address discovery (#1376), and it would probably be better
to update the set of proxyURLs dynamically whenever we fetch the new
state of the cluster; but it needs a bit more thinking to have this done
in a clean way with the proxy interface.

Example in Procfile works again.
2014-10-24 15:54:12 -07:00
Yicheng Qin 49a68adcf1 Merge pull request #1386 from unihorn/184
etcdserver: update member attribute when apply request
2014-10-24 12:46:11 -07:00
Yicheng Qin ea0bff80c0 etcdserver: update member attribute when apply request 2014-10-24 12:43:53 -07:00
Yicheng Qin 08593bcdf6 etcdserver: support newly-join member bootstrap 2014-10-24 12:38:44 -07:00
Yicheng Qin 4d80f01201 etcdserver: Cluster.IsIDremoved -> Cluster.IsIDRemoved 2014-10-23 14:29:58 -07:00
Yicheng Qin 8eee8c260e etcdserver: rebase on master and code clean 2014-10-23 13:58:55 -07:00
Yicheng Qin f8b8bdeb17 etcdserver: use path.Join for member key in cluster 2014-10-23 13:27:54 -07:00
Yicheng Qin 3d243baacd etcdserver: generate id when new cluster 2014-10-23 13:27:54 -07:00
Yicheng Qin 67412e07f8 etcdserver: MemberFromName -> MemberByName 2014-10-23 13:27:54 -07:00
Yicheng Qin 89572b5fd7 etcdserver: refactor cluster and clusterStore
Integrate clusterStore into cluster, and let cluster become the source of
cluster info.
2014-10-23 13:27:54 -07:00
Brandon Philips ab90369f9e etcdserver: use hex for cluster and machine id
Continue using hex everywhere. Including here.

TODO: cleanup the printing of the structs which currently have decimal
to/from:

`{Type:MsgAppResp To:9973738105406047488 From:17050684879817348455 T...`
2014-10-22 16:24:50 -07:00
Barak Michener e42d65da12 etcdserver: Check the initial cluster settings after checking if the WAL exists 2014-10-22 18:16:43 -04:00
Barak Michener 13656eb4e7 Merge pull request #1340 from barakmich/better_ids2
etcdserver: Calculate IDs based on PeerURLs and --initial-cluster-name
2014-10-22 14:49:49 -04:00
Yicheng Qin 89b032cd69 etcdserver: Member.storeKey -> memberStoreKey 2014-10-22 11:09:36 -07:00
Yicheng Qin 7498234e40 etcdserver: record removed member to check incoming message 2014-10-22 11:09:35 -07:00
Barak Michener 502a3c2460 Refactor Cluster to hold and add members. 2014-10-22 13:52:42 -04:00
Barak Michener 1ca7c031ff first round of comments
Conflicts:
	etcdserver/config.go
	etcdserver/config_test.go
	etcdserver/server.go
	main.go
2014-10-22 13:49:54 -04:00
Barak Michener 456d1ebcae etcdserver: Calculate IDs for nodes solely on PeerURLs
Removes the notion of name being anything more than advisory or
command-line grouping, and adds checks for bootstrapping the command
line. IDs are consistent if the URLs are consistent.
2014-10-22 13:49:54 -04:00
Yicheng Qin e2b6a4fc4c etcdserver: const XXXDir -> StoreXXXPrefix
and code clean
2014-10-21 16:10:49 -07:00
Yicheng Qin 2ff3cac653 etcdserver/etcdhttp: store location adjustment
Detailed adjustment:
/_etcd/machines/* -> /0/members/*
/* -> /1/*

And it keeps key path returned to user the same as before.
2014-10-21 16:10:19 -07:00
Xiang Li 894e678ad6 etcdserver: checking clusterID 2014-10-21 11:05:24 -07:00
Xiang Li a44849deec Merge pull request #1286 from coreos/clusterid
*: generate clusterid
2014-10-20 19:07:03 -07:00
Xiang Li 0fd28169c8 etcdserver: use id,cid 2014-10-20 16:35:41 -07:00
Xiang Li dc68dc9ebd etcdserver: add a todo for clusterid generation 2014-10-20 16:26:31 -07:00
Yicheng Qin 92230cee63 Merge pull request #1333 from unihorn/172
etcdserver/raft: remove msgDenied, removedNodes, shouldStop
2014-10-20 16:04:09 -07:00
Yicheng Qin e200d2a8e2 etcdserver/raft: remove msgDenied, removedNodes, shouldStop
The future plan is to do all these in etcdserver level.
2014-10-20 15:13:18 -07:00
Xiang Li ea6bcacfe4 *: generate clusterid 2014-10-20 15:00:54 -07:00
Yicheng Qin f7a0d5387b etcdserver: stop server gracefully 2014-10-20 14:43:21 -07:00
Yicheng Qin 058537f34a main: add basic functional test 2014-10-20 14:43:21 -07:00
Yicheng Qin dcaa7f0a37 Merge pull request #1328 from unihorn/169
skip initial-cluster check when reboot
2014-10-20 13:06:24 -07:00
Yicheng Qin 17382ec905 etcdserver: skip initial-cluster check when reboot
If etcd is provided with data-dir that has data, it will not use
initial-cluster, and initial-cluster could be set to be empty.
2014-10-20 12:41:07 -07:00
Jonathan Boulle 7a4d42166b *: add license header to all source files 2014-10-17 15:41:22 -07:00
Jonathan Boulle c28907ba95 etcdserver: fix race and improve stats interfaces 2014-10-16 23:05:48 -07:00
Jonathan Boulle c30b82b596 etcdserver: fix data race in retrieving self stats 2014-10-16 23:05:48 -07:00
Jonathan Boulle 97ae531eda etcdserver: split out storestats and serverstats 2014-10-16 10:43:49 -07:00
Jonathan Boulle fedb67a71a etcdserver: fix TODO and change to base 16 2014-10-16 10:43:49 -07:00
Jonathan Boulle 8168fed825 etcdserver: add ServerStats and LeaderStats
This adds the remaining two stats endpoints: `/v2/stats/self`, for
various statistics on the EtcdServer, and `/v2/stats/leader`, for
statistics on a leader's followers.

By and large most of the stats code is copied across from 0.4.x, updated
where necessary to integrate with the new decoupling of raft from
transport.

This does not satisfactorily resolve the question of name vs ID. In the
old world, names were unique in the cluster and transmitted over the
wire, so they could be used safely in all statistics. In the new world,
a given EtcdServer only knows its own name, and it is instead IDs that
are communicated among the cluster members. Hence in most places here we
simply substitute a string-encoded ID in place of name, and only where
possible do we retain the actual given name of the EtcdServer.
2014-10-16 10:43:44 -07:00
Jonathan Boulle 0a8721a708 etcdserver: expose store statistics 2014-10-16 10:43:23 -07:00
Yicheng Qin 8cd6030a1d etcdserver: add checking when apply conf change 2014-10-16 09:49:26 -07:00
Yicheng Qin a8a1d4fd93 Merge pull request #1299 from unihorn/162
etcdhttp: add PUT and DELETE on /v2/admin/members/
2014-10-15 10:52:11 -07:00
Xiang Li 828accf07b etcdserver: better logging 2014-10-15 13:07:34 +08:00
Xiang Li f98fbbfc14 *: proto refactoring 2014-10-14 21:07:23 +08:00
Jonathan Boulle 4183b69e12 *: move from third_party to Godep 2014-10-14 00:37:52 -07:00
Yicheng Qin 31264e7eb5 etcdhttp: add PUT and DELETE on /v2/admin/members/ 2014-10-13 22:36:06 -07:00
Xiang Li 6fc0b1977b etcdserver: Id -> ID 2014-10-14 11:30:49 +08:00
Xiang Li b53b74733a etcdserver: add s.apply 2014-10-14 11:30:49 +08:00
Xiang Li 30c7a7f2dd etcdserver: add shouldDiscover 2014-10-14 11:29:00 +08:00
Xiang Li a85ec90d68 etcdserver: clean NewServer 2014-10-14 11:29:00 +08:00
Yicheng Qin f693c6ddf2 etcdserver: apply bootstrap conf change 2014-10-13 11:22:23 -07:00
Yicheng Qin 0319b033ea etcdserver/raft: set context for bootstrap addnode entries 2014-10-13 11:22:23 -07:00
Xiang Li 3516cc3ee5 *: change all ID to uint64 2014-10-12 08:38:48 +08:00
Yicheng Qin f16a272898 Merge pull request #1234 from unihorn/152
etcdserver: save PeerURLs and Others into two different keys
2014-10-10 12:21:32 -07:00
Yicheng Qin 447caf1afc etcdserver/wal: record info at the head of WAL file 2014-10-10 11:57:09 -07:00
Yicheng Qin 77271b0663 etcdserver: split Member into RaftAttributes and Attributes
The split helps to save them in different key paths, because they have
distinct life cycle on update.
2014-10-09 20:28:00 -07:00
Xiang Li 8bbbaa88b2 *: raft related int64 -> uint64 2014-10-09 14:29:21 +08:00
Jonathan Boulle 682008724d etcdserver: add discovery warning message 2014-10-08 16:09:12 -07:00
Yicheng Qin 8e8719f6ac etcdserver: Create, Delete -> Add, Remove 2014-10-07 16:21:58 -07:00
Yicheng Qin cdea98d434 etcdserver: skip bootstrap addNode entry
This is a hack to make etcd work normally.
2014-10-07 11:11:31 -07:00
Yicheng Qin 6a62621695 etcdserver: update panic info 2014-10-07 11:04:50 -07:00
Yicheng Qin 3859297225 etcdserver: check id match 2014-10-07 11:04:50 -07:00
Yicheng Qin d051af4d3d etcdserver: apply config change on cluster store 2014-10-07 11:04:50 -07:00
Xiang Li 5587e0d73f raft: compact takes index and nodes parameters
Before this commit, compact always compact log at current appliedindex of raft.
This prevents us from doing non-blocking snapshot since we have to make snapshot
and compact atomically. To prepare for non-blocking snapshot, this commit make
compact supports index and nodes parameters. After completing snapshot, the applier
should call compact with the snapshot index and the nodes at snapshot index to do
a compaction at snapsohot index.
2014-10-07 16:03:11 +08:00
Barak Michener 39e0a0cd0a Merge pull request #1249 from barakmich/sanity_check
Split config into a separate file and add sanity check and test
2014-10-06 20:19:23 -04:00
Yicheng Qin 3ca3c9ad4c Merge pull request #1245 from unihorn/155
main/raft: write addNode ConfChange entries in log when start raft
2014-10-06 17:00:28 -07:00
Barak Michener 120b088723 Split config into a separate file and add sanity check and test 2014-10-06 19:42:13 -04:00
Yicheng Qin 45ebfb4217 raft: refine initial entries logic in StartNode 2014-10-06 16:06:01 -07:00
Yicheng Qin c15c3eab4c etcdserver: move int64Slice into pkg/types/ 2014-10-06 15:12:02 -07:00
Brandon Philips e2d8037ded main: use initial-cluster and initial-cluster-state flags
In preperation for adding the ability to join a machine to an existing
cluster force the user to specify whether they expect this to me a new
cluster or an active one.

The error for not specifying the initial-cluster-state is:
```
etcd: initial cluster state unset and no wal found
```
2014-10-06 14:59:25 -07:00
Yicheng Qin 314d425718 main/raft: write addNode ConfChange entries in log when start raft 2014-10-06 14:33:12 -07:00
Xiang Li 9e3d045b2b *:discovery hook up 2014-10-06 10:12:42 -07:00
Jonathan Boulle 1308c3e809 etcdserver: unexport EtcdServer fields where possible 2014-10-03 13:34:56 -07:00
Jonathan Boulle e5f5fcff48 etcdserver: collapse dir configuration to DataDir 2014-10-03 13:34:49 -07:00
Jonathan Boulle af6b29f291 main: move server configuration to etcdserver package 2014-10-03 13:34:49 -07:00
Yicheng Qin e4a6c9651a raft: add removed
The usage of removed:
1. tell removed node about its removal explicitly using msgDenied
2. prevent removed node disrupt cluster progress by launching leader election

It is set when apply node removal, or receive msgDenied.
2014-10-01 14:57:38 -07:00
Brandon Philips 83137f9eba pkg/types: introduce a URLs type
Cleanup the usage of URLs into its own type so we don't have to use a
FlagValue everywhere we have a list of URLs.
2014-10-01 14:41:01 -07:00
Yicheng Qin 073eb7677d etcdserver: move grep member logic into publish func 2014-10-01 11:50:47 -07:00
Yicheng Qin cbc84bc70e etcdserver: minimize input info 2014-10-01 11:47:17 -07:00
Yicheng Qin a40a270e19 etcdserver: publish self info when start 2014-10-01 11:47:17 -07:00
Yicheng Qin 34547229a6 etcdserver: fix left Id -> ID 2014-09-29 14:14:45 -07:00
Yicheng Qin 0e8345aa73 Merge pull request #1143 from unihorn/136
*: Id -> ID for protobuf types
2014-09-29 13:58:02 -07:00
Yicheng Qin 89077167c3 etcdserver: add publish func 2014-09-29 10:13:06 -07:00
Yicheng Qin 1d5d2e3726 *: Id -> ID for protobuf types
We use ID instead of Id in this project based on golang conventions.
2014-09-26 11:49:30 -07:00
Brandon Philips 5e3fd6ee3f etcdserver: introduce the cluster and member
This introduces two new concepts: the cluster and the member.

Members are logical etcd instances that have a name, raft ID, and a list
of peer and client addresses.

A cluster is made up of a list of members.
2014-09-25 18:04:10 -07:00
Jonathan Boulle 9b3478218e etcdserver: add RaftIndex and RaftTerm 2014-09-25 16:46:24 -07:00
Jonathan Boulle c28fd92d10 etcdserver: extend storeRecorder to test parameters 2014-09-24 14:33:51 -07:00
Jonathan Boulle 172a32e5e3 etcdserver: correct timeout and streaming handling
This reintroduces the 'stream' parameter to support long-lived watch
sessions. These sessions respect a server timeout (set to 5 minutes by
default).
2014-09-24 11:27:36 -07:00
Yicheng Qin bc7b0108dc raft: ConfigChange -> ConfChange 2014-09-23 12:02:44 -07:00
Yicheng Qin d92931853e raft: Config -> ConfigChange
Configure -> ProposeConfigChange
AddNode, RemoveNode -> ApplyConfigChange
2014-09-22 23:39:53 -07:00
Yicheng Qin ec8f493fde raft: refine comments for Configure 2014-09-22 15:44:47 -07:00
Yicheng Qin dc36ae7058 raft: use pb.Config instead of []byte for Configure 2014-09-22 15:44:47 -07:00
Yicheng Qin 4203569da2 etcdserver: use ConfigType in protobuf 2014-09-22 15:44:46 -07:00
Yicheng Qin b82d70871f raft: use EntryType in protobuf 2014-09-22 15:44:46 -07:00
Yicheng Qin abdb2cad15 etcdserver: Config.Id -> Config.ID 2014-09-22 15:44:46 -07:00
Yicheng Qin aaffb9eb78 etcdserver: add AddNode, RemoveNode
AddNode and RemoveNode is used to propose config change to the cluster.
If succeeds, it will add/remove node from the cluster.
2014-09-22 15:44:46 -07:00
Brian Waldon 7d1126fb35 etcdserver: init time.Time only if Expiration > 0
golang's concept of "zero" time does not align with the zero value of
a unix timestamp. Initializing time.Time with a unix timestamp of 0
makes time.Time.IsZero fail. Solve this by initializing time.Time only
if we care about the time.
2014-09-22 15:31:31 -07:00
Jonathan Boulle e08df4c8d2 store: fix recursive/dir arguments 2014-09-22 14:44:01 -07:00
Brian Waldon 91003cb994 *: s/prevExists/prevExist/ 2014-09-18 21:25:08 -07:00
Jonathan Boulle a42d52482c etcdserver: use simple PRNG for GenID 2014-09-17 15:29:14 -07:00
Xiang Li ab61a8aa9a *: init for on disk snap support 2014-09-17 13:56:12 -07:00
Yicheng Qin 023dc7cba2 etcdserver: add SYNC request 2014-09-16 13:42:03 -07:00
Jonathan Boulle 8a5ab2ec06 etcdserver: introduce Server interface
This changes etcdserver.Server to an interface, with the former Server
(now "EtcdServer") becoming the canonical/production implementation.
This will facilitate better testing of the http server et al with mock
implementations of the interface.
It also more clearly defines the boundary for users of the Server.
2014-09-15 15:11:01 -07:00
Yicheng Qin 6cd4434ff3 server: add unit tests
Make test coverage >= 90%
2014-09-15 13:16:48 -07:00
Jonathan Boulle f7444ff300 store: convert Watch to interface 2014-09-09 11:17:53 -07:00
Blake Mizerany d844377ca6 *: vendor context 2014-09-03 20:06:16 -07:00
Xiang Li 2ba57ee75d *: remove unused pkgs 2014-09-03 09:20:20 -07:00