Commit Graph

3968 Commits (20e2c8f431cbf12ab46bb701573455ad0cbabea5)

Author SHA1 Message Date
Xiang Li 35e6df6d0a integration: longer request timeout for slow travis 2014-11-23 22:53:05 -08:00
Xiang Li da1ff2d2bb pkg/transport: add timeout dailer and timeout listener 2014-11-23 21:18:34 -08:00
Xiang Li 68e79868cc Merge pull request #1768 from xiang90/batcher
rafthttp: add batcher
2014-11-23 12:08:00 -08:00
Xiang Li 91bfead9e9 rafthttp: add batcher
After we enable streaming, there will be a large amount of msgAppResp due to high
rate msgApp. We should batch msgAppResp in a meanful way.
2014-11-23 12:05:42 -08:00
Ben Darnell 9ddd8ee539 Rename Storage.HardState back to InitialState and include ConfState.
This fixes integration/migration_test.go (and highlights the fact that
we need some more raft-level testing of restoring from snapshots).
2014-11-21 17:22:20 -05:00
Ben Darnell 03c8881e35 Fix TestSlowNodeRestore 2014-11-21 16:40:41 -05:00
Ben Darnell 0d680d0e6b Merge remote-tracking branch 'coreos/master' into merge
* coreos/master:
  rafthttp: fix import
  raft: should not decrease match and next when handling out of order msgAppResp
  Fix migration to allow snapshots to have the right IDs
  add snapshotted integration test
  fix test import loop
  fix import loop, add set to types, and fix comments
  etcdserver: autodetect v0.4 WALs and upgrade them to v0.5 automatically
  wal: add a bench for write entry
  rafthttp: add streaming server and client
  dep: use vendored imports in codegangsta/cli
  dep: bump golang.org/x/net/context

Conflicts:
	etcdserver/server.go
	etcdserver/server_test.go
	migrate/snapshot.go
2014-11-21 15:40:11 -05:00
Ben Darnell 30690d15d9 Re-enable a few tests I had missed.
Fix integration test for the change to log entry zero.

Increase test timeouts since integration tests often take
longer than 10s for me.
2014-11-21 15:27:17 -05:00
Xiang Li 66c30f28d6 Merge pull request #1761 from xiang90/fix_raft
raft: should not decrease match and next when handling out of order msgAppResp
2014-11-20 18:12:15 -08:00
Xiang Li edcdffe11e Merge pull request #1762 from xiang90/fix_import
rafthttp: fix import
2014-11-20 18:10:49 -08:00
Xiang Li 264a63be80 rafthttp: fix import 2014-11-20 18:05:45 -08:00
Xiang Li 063c5c77a0 raft: should not decrease match and next when handling out of order msgAppResp 2014-11-20 17:58:23 -08:00
Brian Waldon c0fb1c8a00 Merge pull request #1755 from bcwaldon/golang.org-deps
Switch to golang.org/x/net/context
2014-11-20 16:26:14 -08:00
Barak Michener 5139257b8d Merge pull request #1743 from barakmich/auto_upgrade
etcdserver: autodetect v0.4 WALs and upgrade them to v0.5 automatically
2014-11-20 17:20:34 -05:00
Barak Michener ce82a3e7ad Fix migration to allow snapshots to have the right IDs 2014-11-20 16:49:34 -05:00
Xiang Li 53fbf0f333 Merge pull request #1760 from xiang90/wal
wal: add a bench for write entry
2014-11-20 13:16:37 -08:00
Barak Michener 2d5ccf12ef add snapshotted integration test 2014-11-20 15:38:08 -05:00
Barak Michener d1e7fee3ca fix test import loop 2014-11-20 15:38:08 -05:00
Barak Michener 59a0c64e9f fix import loop, add set to types, and fix comments 2014-11-20 15:38:08 -05:00
Barak Michener 78ea3335bf etcdserver: autodetect v0.4 WALs and upgrade them to v0.5 automatically 2014-11-20 15:38:08 -05:00
Xiang Li d0dd205b0e wal: add a bench for write entry 2014-11-20 12:07:17 -08:00
Yicheng Qin aca195f3ad Merge pull request #1746 from yichengq/229
rafthttp: streaming server and client
2014-11-20 11:42:00 -08:00
Yicheng Qin 9d53b94546 rafthttp: add streaming server and client 2014-11-20 11:34:50 -08:00
Brian Waldon da5538b8c7 dep: use vendored imports in codegangsta/cli 2014-11-20 10:19:13 -08:00
Brian Waldon 9a728a127a dep: bump golang.org/x/net/context
Move from code.google.com/p/go.net/context to
golang.org/x/net/context before bumping to latest.
2014-11-20 10:19:12 -08:00
Ben Darnell b29240baf0 Merge remote-tracking branch 'coreos/master' into merge
* coreos/master:
  scripts: build-docker tag and use ENTRYPOINT
  scripts: build-release add etcd-migrate
  create .godir
  raft: optimistically increase the next if the follower is already matched
  raft: add handleHeartbeat handleHeartbeat commits to the commit index in the message. It never decreases the commit index of the raft state machine.
  rafthttp: send takes raft message instead of bytes
  *: add rafthttp pkg into test list
  raft: include commitIndex in heartbeat
  rafthttp: move server stats in raftHandler to etcdserver
  *: etcdhttp.raftHandler -> rafthttp.RaftHandler
  etcdserver: rename sender.go -> sendhub.go
  *: etcdserver.sender -> rafthttp.Sender

Conflicts:
	raft/log.go
	raft/raft_paper_test.go
2014-11-19 17:05:16 -05:00
Ben Darnell 355ee4f393 raft: Integrate snapshots into the raft.Storage interface.
Compaction is now treated as an implementation detail of Storage
implementations; Node.Compact() and related functionality have been
removed. Ready.Snapshot is now used only for incoming snapshots.

A return value has been added to ApplyConfChange to allow applications
to track the node information that must be stored in the snapshot.

raftpb.Snapshot has been split into Snapshot and SnapshotMetadata, to
allow the full snapshot data to be read from disk only when needed.

raft.Storage has new methods Snapshot, ApplySnapshot, HardState, and
SetHardState. The Snapshot and HardState parameters have been removed
from RestartNode() and will now be loaded from Storage instead.
The only remaining difference between StartNode and RestartNode is that
the former bootstraps an initial list of Peers.
2014-11-19 16:40:26 -05:00
Xiang Li b50f331558 Merge pull request #1744 from xiang90/next
raft: optimistically increase the next if the follower is already matched
2014-11-19 13:21:11 -08:00
Brandon Philips 12aaf046d7 Merge pull request #1752 from philips/include-etcd-migrate-in-releases
scripts: build-release add etcd-migrate
2014-11-19 12:13:21 -08:00
Brandon Philips f08df9e0f3 Merge pull request #1753 from philips/fixup-build-docker
scripts: build-docker tag and use ENTRYPOINT
2014-11-19 12:13:08 -08:00
Brandon Philips 00df13138e scripts: build-docker tag and use ENTRYPOINT
Use ENTRYPOINT so people can specify flags to etcd without providing the
binary.

Thanks to @hugod in IRC for pointing this out.
2014-11-19 12:11:27 -08:00
Brandon Philips d2e36a9535 scripts: build-release add etcd-migrate
this tool is only temporary for the alphas but make sure it makes it in.
2014-11-19 12:10:06 -08:00
Brandon Philips 893fb3b890 Merge pull request #1747 from proppy/master
create .godir
2014-11-18 16:00:07 -08:00
Johan Euphrosine 03bacc1984 create .godir 2014-11-18 15:01:57 -08:00
Xiang Li 4c1fd07311 raft: optimistically increase the next if the follower is already matched
This is useful since we want to pipeline the appendEntry requests. Without
enabling optimistic increasing, the second pipelining appendEntry request
will include the entries the first one has already sent out. We decrease
the next directly to match if the leader receives a rejection for a matched
follower. This happens if one pipelining request get lost and following ones
arrives at the follower.
2014-11-18 13:41:38 -08:00
Xiang Li e07ef6991c Merge pull request #1740 from xiang90/handleheartbeat
raft: add handleHeartbeat
2014-11-18 13:25:12 -08:00
Ben Darnell 46ee58c6f0 raft: Rename ErrSnapshotRequired To ErrCompacted. 2014-11-18 13:15:10 -05:00
Yicheng Qin f94ff96c69 Merge pull request #1739 from yichengq/230
rafthttp: send takes raft message instead of bytes
2014-11-18 10:11:05 -08:00
Xiang Li bd4cfa2a07 raft: add handleHeartbeat
handleHeartbeat commits to the commit index in the message. It never decreases the
commit index of the raft state machine.
2014-11-18 08:34:06 -08:00
Yicheng Qin 1a72143ecb rafthttp: send takes raft message instead of bytes
This gives streaming mechanism the chance to assemble and disassemble
raft messages.
2014-11-17 22:39:53 -08:00
Xiang Li 6cac631a0d Merge pull request #1738 from yichengq/231
*: add rafthttp pkg into test list
2014-11-17 17:14:40 -08:00
Yicheng Qin 04d416291a *: add rafthttp pkg into test list 2014-11-17 16:45:54 -08:00
Xiang Li 1635844ebd Merge pull request #1737 from xiang90/include_commit
raft: include commitIndex in heartbeat
2014-11-17 16:26:07 -08:00
Xiang Li b93d87f17f raft: include commitIndex in heartbeat 2014-11-17 16:19:28 -08:00
Yicheng Qin a2c568a144 Merge pull request #1669 from yichengq/215
*: add rafthttp as a separate package
2014-11-17 16:14:59 -08:00
Yicheng Qin f24e214ee5 rafthttp: move server stats in raftHandler to etcdserver 2014-11-17 16:02:20 -08:00
Yicheng Qin 5dc5f8145c *: etcdhttp.raftHandler -> rafthttp.RaftHandler 2014-11-17 15:52:24 -08:00
Yicheng Qin 3fcc011717 etcdserver: rename sender.go -> sendhub.go 2014-11-17 15:35:15 -08:00
Yicheng Qin 84fbf7aab5 *: etcdserver.sender -> rafthttp.Sender 2014-11-17 15:35:10 -08:00
Ben Darnell 300c5a2001 Merge remote-tracking branch 'coreos/master' into log-storage-interface
* coreos/master: (21 commits)
  etcdserver: refactor ValidateClusterAndAssignIDs
  integration: add integration test for remove member
  integration: add test for member restart
  version: bump to alpha.3
  etcdserver: add buffer to the sender queue
  *: gracefully stop etcdserver
  Fix up migration tool, add snapshot migration
  etcd4: migration from v0.4 -> v0.5
  etcdserver: export Member.StoreKey
  etcdserver: recover cluster when receiving newer snapshot
  etcdserver: check and select committed entries to apply
  etcdserver: recover from snapshot before applying requests
  raft: not set applied when restored from snapshot
  sender: support elegant stop
  etcdserver: add StopNotify
  etcdserver: fix TestDoProposalStopped test
  etcdserver: minor cleanup
  etcdserver: validate new node is not registered before in best effort
  etcdserver: fix server.Stop()
  *: print out configuration when necessary
  ...

Conflicts:
	etcdserver/server.go
	etcdserver/server_test.go
	raft/log.go
2014-11-17 18:28:24 -05:00