tc

Each card sends through a queueing discipline (qdisc). netem adds delay, jitter, loss, duplication or reordering, which is how netlab gives links their character. htb shapes: it holds traffic to a rate and splits it into classes, each with a guaranteed rate, a ceil it may borrow up to, and a prio; filters (tc filter … u32 match ip dsfield …) sort packets into classes, for example by DSCP mark. sfq gives each flow its own small queue, served in turn. tbf, the token bucket filter, is the plain shaper: one rate, one queue. A police action on a filter drops what exceeds a rate instead of queueing it. tc -s adds counters: sent, dropped, and the backlog waiting, which shows where a queue really builds.

tc command usage

tc [-s] qdisc|class|filter show [dev CARD]
tc qdisc add|change|replace|del dev CARD root|parent ID [handle ID] netem|htb|tbf|sfq|prio …
tc class add|change dev CARD parent ID classid ID htb rate R [ceil R] [prio N]
tc filter add dev CARD parent ID protocol ip u32 match ip dsfield|src|dst … flowid ID

Options

show dev eth0

the qdiscs on eth0

-s

statistics: sent, dropped, backlog

netem delay 40ms 10ms

add 40 ms ± 10 ms each way

loss 2%

drop 2% of packets

htb default 20

a shaper whose unmatched traffic goes to class 1:20

rate 7500kbit ceil 8mbit

a class's guaranteed rate, and the most it may borrow

match ip dsfield 0xb8 0xfc

packets marked DSCP EF (the mask leaves out the ECN bits)

tbf rate 2mbit burst 20k latency 300ms

shape to 2 Mbit/s; a packet may wait up to 300 ms for a token

police rate 2mbit burst 20k drop

drop what goes over 2 Mbit/s, allowing bursts of 20 KB

del dev eth0 root

remove everything, back to the default

tc examples

tc -s qdisc show dev eth0

Shows the queue with its counters: packets sent, dropped, and waiting.

tc qdisc add dev eth0 root netem delay 40ms 10ms loss 2%

Adds 40 ms ± 10 ms of delay and 2% loss to everything leaving eth0.

tc qdisc add dev eth0 root handle 1: htb default 1 && tc class add dev eth0 parent 1: classid 1:1 htb rate 7500kbit

Shapes everything leaving eth0 to 7.5 Mbit/s.

tc qdisc add dev eth0 parent 1:1 sfq perturb 10

Shares that class fairly between flows.

tc filter add dev eth0 parent 1: protocol ip u32 match ip dsfield 0xb8 0xfc flowid 1:10

Sends EF-marked packets to class 1:10.

tc -s class show dev eth0

Shows each class with what it has sent and what waits in it.

tc qdisc add dev eth0 root tbf rate 3600kbit burst 5k latency 100ms

Shapes everything leaving eth0 to 3.6 Mbit/s with a token bucket: the excess waits, up to 100 ms.

tc -s filter show dev eth0 parent ffff:

Lists the ingress filters with their counters: how many packets a policer dropped.

tc qdisc del dev eth0 root

Removes the added queues, so eth0 sends normally again.

Reading the output

$ tc -s qdisc show dev eth0
qdisc netem 806c: root refcnt 15 limit 1000 delay 10ms rate 8Mbit
 Sent 10581372 bytes 7042 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 461770b 153p requeues 0

• netem … delay 10ms rate 8Mbit: this card adds 10 ms and sends at most 8 Mbit/s. Replies come back through the other side's card, with its own qdisc.
• backlog 461770b 153p: 153 packets wait in this queue right now. That is where the delay under load comes from.
• qdisc noqueue and nothing else: no shaping on this card, so nothing ever waits in it.

Pitfalls

• tc acts on what a card sends only. To slow both directions, set it on both ends; to shape what comes in, police it or shape it on the other side.
• Priorities only matter where packets wait. On a fast card in front of a slow link, shape below the link's rate first, or the queue forms elsewhere.
• add fails if a root qdisc is already there: use change or replace.

Run it on real machines