diff --git a/Documentation/tuning.md b/Documentation/tuning.md index dbd5c6b3e..f1cdc38cc 100644 --- a/Documentation/tuning.md +++ b/Documentation/tuning.md @@ -71,4 +71,23 @@ $ etcd --snapshot-count=5000 $ ETCD_SNAPSHOT_COUNT=5000 etcd ``` +## Network + +If the etcd leader serves a large number of concurrent client requests, it may delay processing follower peer requests due to network congestion. This manifests as send buffer error messages on the follower nodes: + +``` +dropped MsgProp to 247ae21ff9436b2d since streamMsg's sending buffer is full +dropped MsgAppResp to 247ae21ff9436b2d since streamMsg's sending buffer is full +``` + +These errors may be resolved by prioritizing etcd's peer traffic over its client traffic. On Linux, peer traffic can be prioritized by using the traffic control mechanism: + +``` +tc qdisc add dev eth0 root handle 1: prio bands 3 +tc filter add dev eth0 parent 1: protocol ip prio 1 u32 match ip sport 2380 0xffff flowid 1:1 +tc filter add dev eth0 parent 1: protocol ip prio 1 u32 match ip dport 2380 0xffff flowid 1:1 +tc filter add dev eth0 parent 1: protocol ip prio 2 u32 match ip sport 2739 0xffff flowid 1:1 +tc filter add dev eth0 parent 1: protocol ip prio 2 u32 match ip dport 2739 0xffff flowid 1:1 +``` + [ping]: https://en.wikipedia.org/wiki/Ping_(networking_utility)