Diagnosing issues in a distributed application is hard.
etcd will help as much as it can - just enable these debug features using the CLI flag `-trace=*` or the config option `trace=*`.
## Logging
Log verbosity can be increased to the max using either the `-vvv` CLI flag or the `very_very_verbose=true` config option.
The only supported logging mode is to stdout.
## Metrics
etcd itself can generate a set of metrics.
These metrics represent many different internal data points that can be helpful when debugging etcd servers.
#### Metrics reference
Each individual metric name is prefixed with `etcd.<NAME>`, where \<NAME\> is the configured name of the etcd server.
*`timer.appendentries.handle`: amount of time a peer takes to process an AppendEntriesRequest from the POV of the peer itself
*`timer.peer.<PEER>.heartbeat`: amount of time a peer heartbeat operation takes from the POV of the leader that initiated that operation for peer \<PEER\>
*`timer.command.<COMMAND>`: amount of time a given command took to be processed through the local server's raft state machine. This does not include time waiting on locks.
#### Fetching metrics over HTTP
Once tracing has been enabled on a given etcd server, all metric data is available at the server's `/debug/metrics` HTTP endpoint (i.e. `http://127.0.0.1:4001/debug/metrics`).
Executing a GET HTTP command against the metrics endpoint will yield the current state of all metrics in the etcd server.
#### Sending metrics to Graphite
etcd supports [Graphite's Carbon plaintext protocol](https://graphite.readthedocs.org/en/latest/feeding-carbon.html#the-plaintext-protocol) - a TCP wire protocol designed for shipping metric data to an aggregator.
To send metrics to a Graphite endpoint using this protocol, use of the `-graphite-host` CLI flag or the `graphite_host` config option (i.e. `graphite_host=172.17.0.19:2003`).
See an [example graphite deploy script](https://github.com/coreos/etcd/contrib/graphite).
#### Generating additional metrics with Collectd
[Collectd](http://collectd.org/documentation.shtml) gathers metrics from the host running etcd.
While these aren't metrics generated by etcd itself, it can be invaluable to compare etcd's view of the world to that of a separate process running next to etcd.
See an [example collectd deploy script](https://github.com/coreos/etcd/contrib/collectd).
## Profiling
etcd exposes profiling information from the Go pprof package over HTTP.
**NOTE**: In the following examples you need to ensure that the `./bin/etcd` is identical to the `./bin/etcd` that you are targeting (same git hash, arch, platform, etc).
