Commit Graph

179 Commits (9665cda7c1bd2d97a901bd4d311dffe359fe9bb4)

Author SHA1 Message Date
Yicheng Qin df83af944b Merge pull request #3384 from yichengq/fix-shadow
test: use go vet shadow feature instead of go-nyet
2015-08-27 14:27:57 -07:00
Yicheng Qin 92cd24d5bd *: fix govet shadow check failure 2015-08-27 14:15:30 -07:00
Xiang Li b2d33e6dcb Merge pull request #3382 from xiang90/env
pkg/flags: print out evn usage information
2015-08-27 13:36:55 -07:00
Matt Keller 327632014e cors: Removed new(?) header from test, resolving failure
"X-Content-Type-Options" was being autoadded, but none of the
test maps took it into account. I saw that "Content-Type" was
also being deleted, so I figured that was the best solution
for this as well.
2015-08-27 15:23:14 -04:00
Xiang Li c8f5e03b75 pkg/flags: print out evn usage information 2015-08-27 12:08:31 -07:00
Yicheng Qin 57e88465bf pkg/transport: print ClientCertAuth in TLSInfo.String()
It is good to print it in debug output:

```
21:56:12 etcd1 | 2015-08-25 21:56:12.162406 I | etcdmain: peerTLS: cert
= certs/etcd1.pem, key = certs/etcd1-key.pem, ca = , trusted-ca =
certs/ca.pem, client-cert-auth = true
```
2015-08-25 21:53:52 -07:00
Yicheng Qin 6250fed8a8 Merge pull request #3096 from philips/tls-info-debug
pkg/transport: include debug output for trusted-ca
2015-08-25 20:08:19 -07:00
Yicheng Qin fc95ec0cc6 rafthttp: always cancel in-flight request when stop streamReader
This problem is totally fixed at 1.5.

go1.5 adds a Request.Cancel channel, which allows for "race free"
cancellation
(8b4278ffb7).
Our implementation relies on it to always cancel in-flight request.
2015-08-25 08:54:13 -07:00
Cong Ding c09b667d57 *: fix go vet reported issues 2015-08-22 12:19:02 -05:00
Xiang Li 044b23c3ca Merge pull request #3356 from xiang90/travis
*: test gofmt with -s and fix reported issues
2015-08-21 18:59:51 -07:00
Xiang Li 6b23a8131f *: test gofmt with -s and fix reported issues 2015-08-21 18:52:16 -07:00
Yicheng Qin 3c1e6b54b3 pkg/netutil: stop resolving in place
It helps to copy out a and b, and not modify the original a and b.
2015-08-21 12:09:17 -07:00
Yicheng Qin 1c334979cd pkg/netutil: not introduce empty url when converting
It should not make slices with length and append elements at the same
time.
2015-08-21 12:08:17 -07:00
Yicheng Qin 7b871aab41 pkg/netutil: not export resolve and urlsEqual functions
They are only used in this package, so there is no need to public them.
2015-08-21 11:58:37 -07:00
Yicheng Qin b1192e5c48 pkg/netutil: fix false negative comparison
Sort the resolved URLs before DeepEqual, so it will not compare URLs
that may be out of order due to resolution.
2015-08-21 10:15:08 -07:00
Yicheng Qin 8ea3d157c5 Revert "Revert "Treat URLs have same IP address as same""
This reverts commit 3153e635d5.

Conflicts:
	etcdserver/config.go
2015-08-21 09:41:13 -07:00
Xiang Li ff37cc455c pkg/transport: remove home-grown limitedListener 2015-08-20 20:03:27 -07:00
Xiang Li 3ca5482251 pkg/fileutil: treat not support error as nil error in preallocate 2015-08-20 11:15:02 -07:00
Yicheng Qin 4778d780a8 pkg/pathutil: change copyright for path.go
The file only contains the function that is borrowed from std http lib,
so we use their copyright.
2015-08-18 11:48:22 -07:00
Yicheng Qin b5ec7f543a client: use canonical url path in request
The main change is that it keeps the trailing slash. This helps
auth feature to judge path permission accurately.
2015-08-18 08:59:48 -07:00
Yicheng Qin 2c2249dadc Merge pull request #3219 from yichengq/limit-listener
etcdmain: stop accepting client conns when it reachs limit
2015-08-06 12:17:49 -07:00
Yicheng Qin 97923ca3fc etcdmain: close client conns when it exceeds limit
This solves the problem that etcd may fatal because its critical path
cannot get file descriptor resource when the number of clients is too
big. The PR lets the client listener close client connections
immediately after they are accepted when
the file descriptor usage in the process reaches some pre-set limit, so
it ensures that the internal critical path could always get file
descriptor when it needs.

When there are tons to clients connecting to the server, the original
behavior is like this:

```
2015/08/4 16:42:08 etcdserver: cannot monitor file descriptor usage
(open /proc/self/fd: too many open files)
2015/08/4 16:42:33 etcdserver: failed to purge snap file open
default2.etcd/member/snap: too many open files
[halted]
```

Current behavior is like this:

```
2015/08/6 19:05:25 transport: accept error: closing connection,
exceed file descriptor usage limitation (fd limit=874)
2015/08/6 19:05:25 transport: accept error: closing connection,
exceed file descriptor usage limitation (fd limit=874)
2015/08/6 19:05:26 transport: accept error: closing connection,
exceed file descriptor usage limitation (fd limit=874)
2015/08/6 19:05:27 transport: accept error: closing connection,
exceed file descriptor usage limitation (fd limit=874)
2015/08/6 19:05:28 transport: accept error: closing connection,
exceed file descriptor usage limitation (fd limit=874)
2015/08/6 19:05:28 etcdserver: 80% of the file descriptor limit is
used [used = 873, limit = 1024]
```

It is available at linux system today because pkg/runtime only has linux
support.
2015-08-06 12:03:20 -07:00
Xiang Li 01c286ccb6 Merge pull request #3231 from xiang90/fallocate
pkg/fileutil: support perallocate
2015-08-06 10:25:28 -07:00
Xiang Li 39a4b6a5e5 pkg/fileutil: support perallocate 2015-08-06 10:10:58 -07:00
Xiang Li 58503817ec etcdserver: internal request union 2015-08-05 07:47:10 -07:00
Yicheng Qin 6317abf7e4 pkg/transport: fix HTTPS downgrade bug for keepalive listener
If TLS config is empty, etcd downgrades keepalive listener from HTTPS to
HTTP without warning. This results in HTTPS downgrade bug for client urls.
The commit returns error if it cannot listen on TLS.
2015-07-21 12:53:21 -07:00
Xiang Li b8279b3591 types: add len func for urlmaps 2015-07-21 12:53:20 -07:00
Yicheng Qin 1624235bb3 pkg/testutil: extend wait schedule time to 10ms
Waiting 3ms is not long enough for schedule to work well. The test suite
may fail once per 200 times in travis due to this. Extend this to 10ms
to ensure schedule could work. Now it could run 1000 times successfully
in travis.
2015-07-13 09:05:40 -07:00
Brandon Philips 09b9c30beb pkg/transport: include debug output for trusted-ca
since --peer-ca-file is deprecated we need to update the debug output

before:

```
$ etcd ... --peer-cert-file infra1.crt -peer-key-file
 infra1.key.insecure -peer-trusted-ca-file ca.crt --client-cert-auth
etcdmain: peerTLS: cert = infra1.crt, key = infra1.key.insecure, ca =
```

after:

```
$ etcd ... --peer-cert-file infra1.crt -peer-key-file
 infra1.key.insecure -peer-trusted-ca-file ca.crt --client-cert-auth
etcdmain: peerTLS: cert = infra1.crt, key = infra1.key.insecure, ca = , trusted-ca = ca.crt
```
2015-07-04 14:28:18 -07:00
Yicheng Qin 3d4642c2c4 Merge pull request #3059 from yichengq/fix-wait-stress-test
pkg/wait: extend timeout to check closed channel
2015-06-25 11:16:54 -07:00
Yicheng Qin eea7f28be4 pkg/wait: extend timeout to check closed channel
It is possible to trigger the time.After case if the timer went off
between time.After setting the timer for its channel and the time that
select looked at the channel. So it needs to be longer.

refer: https://groups.google.com/forum/#!topic/golang-nuts/1tjcV80ccq8
2015-06-25 10:43:12 -07:00
Yicheng Qin 107263ef9f pkg/fileutil: fix TestPurgeFile
It needs to wait longer for file to be detected and removed sometimes.
2015-06-25 10:09:20 -07:00
Xiang Li 8e7fa9e201 Merge pull request #2976 from yichengq/fix-lock-test
pkg/fileutil: wait longer for relock
2015-06-12 15:20:18 -07:00
Yicheng Qin 7723b91c06 pkg/fileutil: wait longer for relock
multiple cpu running makes it slower, so it waits longer for relock.
2015-06-12 15:17:28 -07:00
Yicheng Qin 75f91bab5c pkg/fileutil: wait longer before checking purge results
multiple cpu running may be slower than single cpu running, so it may
take longer time to remove files.
Increase from 5ms to 20ms to give it enough time.
2015-06-12 14:36:15 -07:00
Xiang Li 8ad7ed321e *:godep log pkg 2015-06-11 14:22:14 -07:00
Xiang Li 4b5dbeff9b pkg/pbutil: use leveled log 2015-06-11 14:19:53 -07:00
Xiang Li 865a5ffc61 pkg/osutil: use leveled log 2015-06-11 14:19:53 -07:00
Xiang Li a45f53986f pkg/netutil: use leveled log 2015-06-11 14:19:52 -07:00
Xiang Li 69819d334a pkg/flags: use leveled log 2015-06-11 14:19:52 -07:00
Xiang Li f64a8214f7 Merge pull request #2952 from xiang90/fileutil
fileutil: use leveled logging
2015-06-10 16:01:24 -07:00
Xiang Li dc87454487 fileutil: return on error and send it to error chan 2015-06-10 15:59:24 -07:00
Xiang Li e2c2f098bc fileutil: use leveled logging 2015-06-10 15:57:59 -07:00
Yicheng Qin 018fb8e6d9 pkg/testutil: ForceGosched -> WaitSchedule
ForceGosched() performs bad when GOMAXPROCS>1. When GOMAXPROCS=1, it
could promise that other goroutines run long enough
because it always yield the processor to other goroutines. But it cannot
yield processor to goroutine running on other processors. So when
GOMAXPROCS>1, the yield may finish when goroutine on the other
processor just runs for little time.

Here is a test to confirm the case:

```
package main

import (
	"fmt"
	"runtime"
	"testing"
)

func ForceGosched() {
	// possibility enough to sched up to 10 go routines.
	for i := 0; i < 10000; i++ {
		runtime.Gosched()
	}
}

var d int

func loop(c chan struct{}) {
	for {
		select {
		case <-c:
			for i := 0; i < 1000; i++ {
				fmt.Sprintf("come to time %d", i)
			}
			d++
		}
	}
}

func TestLoop(t *testing.T) {
	c := make(chan struct{}, 1)
	go loop(c)
	c <- struct{}{}
	ForceGosched()
	if d != 1 {
		t.Fatal("d is not incremented")
	}
}
```

`go test -v -race` runs well, but `GOMAXPROCS=2 go test -v -race` fails.

Change the functionality to waiting for schedule to happen.
2015-06-10 14:37:41 -07:00
Yicheng Qin 3153e635d5 Revert "Treat URLs have same IP address as same"
This reverts commit f8ce5996b0.

etcd no longer resolves TCP addresses passed in through flags,
so there is no need to compare hostname and IP slices anymore.
(for more details: a3892221ee)

Conflicts:
	etcdserver/cluster.go
	etcdserver/config.go
	pkg/netutil/netutil.go
	pkg/netutil/netutil_test.go
2015-05-16 03:21:10 -07:00
Yicheng Qin 256a7cfe8c pkg/wait: fix TestWaitTestStress
The test may fail if two consequent time.Now() returns the same value.
Sleep 1ns to avoid this situation.
2015-05-13 13:41:34 -07:00
Yicheng Qin 032db5e396 *: extract types.Cluster from etcdserver.Cluster
The PR extracts types.Cluster from etcdserver.Cluster. types.Cluster
is used for flag parsing and etcdserver config.

There is no need to expose etcdserver.Cluster public, which contains
lots of etcdserver internal details and methods. This is the first step
for it.
2015-05-12 14:53:11 -07:00
mischief 2e8c932ab0 pkg/fileutil: add plan9 lockfile support 2015-05-11 13:24:01 -07:00
Alexander Kolbasov 39c7060d3b pkg/fileutil: add filelock support for solaris 2015-04-24 12:18:08 -07:00
Yicheng Qin 2141308524 Merge pull request #2631 from yichengq/metrics-fd
etcdserver: metrics and monitor number of file descriptor
2015-04-08 11:28:58 -07:00