Forwarding
Routing table
The list a device uses to decide where each packet goes next.
Each route names a destination prefix and a next hop or an interface. The most specific matching route wins
(longest prefix), and default (0.0.0.0/0) matches everything else. ip route get 10.0.2.10 shows the
decision for one address. Routes are written by hand (static), learned from the connected networks, or learned
from a routing protocol like OSPF or BGP.
How a routing table works, step by step
Every packet a Linux machine sends or forwards goes through the same lookup. Take branchRouter, with a laptop
network 10.0.1.0/24 on one side, two links to hqRouter (10.0.12.0/24, the primary, and 10.0.13.0/24, the
backup), and two servers behind hqRouter, web at 10.0.2.10 and files at 10.0.2.11:
- Pick the table. The kernel walks its policy rules (
ip rule) in order of preference. By default there are three:local(the machine's own addresses),main(whatip routeshows) anddefault(empty). A rule can send some packets to another table, by incoming card, source address or firewall mark. - Find every route that contains the destination. For
10.0.2.11, both10.0.2.0/24 via 10.0.12.3and a host route10.0.2.11/32 via 10.0.13.3match, and so doesdefault(0.0.0.0/0). - Longest prefix wins. The /32 is more specific than the /24, which beats the /0. Packets to the file server
cross the backup link; packets to
10.0.2.10still match only the /24 and take the primary. - Among equal prefixes, the lowest metric wins. Two default routes with metrics 100 and 200 both sit in the
table; the 200 is used only when the 100 is gone. Equal metrics with several next hops (
nexthop via ... nexthop via ...) share the load (ECMP). - Resolve the next hop.
via 10.0.12.3must be on a connected network. The kernel ARPs for it and builds the frame with that MAC; the IP destination stays10.0.2.11. - No match at all means the packet is dropped. A host gets
Network unreachable; a router also sends ICMP net unreachable to the source.
Routes come from three places: the kernel adds one for each address's network (proto kernel scope link), people
add static routes, and routing daemons such as FRR install what OSPF or
BGP learn. When two sources offer the same prefix, the daemon chooses by
administrative distance before anything reaches the kernel. Metric decides only
between routes already in the kernel table.
What it looks like on a Linux machine
In the static routes lab branchRouter gets a /32 for the file server over
the backup link, and ip route get shows the decision for each address:
$ ip route get 10.0.2.11
10.0.2.11 via 10.0.13.3 dev backupLink src 10.0.13.2 uid 0
cache
$ ip route get 10.0.2.10
10.0.2.10 via 10.0.12.3 dev primaryLink src 10.0.12.2 uid 0
cache
Two default routes added by hand with ip route add default via 10.0.12.3 metric 100 and the same via 10.0.13.3
with metric 200 both stay in the table:
$ ip route show default
default via 10.0.12.3 dev primaryLink metric 100
default via 10.0.13.3 dev backupLink metric 200
Extra tables and rules are how one Linux router keeps customers apart in the VRF-lite lab. Each customer's card looks up its own numbered table:
$ ip rule show
0: from all lookup local
100: from all iif officeA lookup 10
200: from all iif officeB lookup 20
32766: from all lookup main
32767: from all lookup default
$ ip route get 192.168.50.10 from 10.0.1.10 iif officeA
192.168.50.10 from 10.0.1.10 via 10.0.2.10 dev serversA table 10
cache iif officeA
ip route show table 10 lists one table; ip route show table all lists them all.
What goes wrong, and how you see it
No route back. branchRouter has 10.0.2.0/24 via 10.0.12.3, but hqRouter has nothing for 10.0.1.0/24.
Pings leave and replies die at hqRouter. Every router on the path needs a route for both directions; the
routing across hops lab shows each router knowing one step more.
A more specific route you forgot. An old /32 or a VPN's routes beat your new /24, so traffic goes the wrong way.
ip route get <address> names the route actually used; reading ip route by eye misses it.
A next hop that is not on a connected network. ip route add 10.0.9.0/24 via 10.0.5.1 on a host without
10.0.5.0/24 fails with Error: Nexthop has invalid gateway.
A duplicate prefix. Adding a second route for the same prefix and metric in one table fails with
RTNETLINK answers: File exists. Use another metric, ip route replace, or, for two customers with the same
addresses, a second table.
A route that stays while the path is dead. A static route remains as long as its card is up, whether or not the next hop answers. Floating statics, tracking or a routing protocol fix it. In the routing across hops lab the middle link is cut, the routes stay, and traceroute shows where the answers stop.
Questions people ask
What is longest prefix match? Choosing the matching route with the most network bits: a /32 beats a /24, which beats the default /0. It lets one specific exception override a broad route. Try the boundaries with the subnet calculator.
What is the difference between metric and administrative distance? Distance ranks sources (connected, static, OSPF, BGP) for the same prefix inside a routing daemon. Metric ranks routes of one source, and in the Linux kernel it is the only tiebreak left.
How do I see the routing table on Linux? ip route for the main table, ip -6 route for IPv6, ip rule for
the rules, ip route get for one address. route -n and netstat -rn are the older views. The
bare machine lab builds a table from nothing.