Concepts / Forwarding

Forwarding

Routing table

Also called route, longest prefix match, ip route

The list a device uses to decide where each packet goes next.

Each route names a destination prefix and a next hop or an interface. The most specific matching route wins (longest prefix), and default (0.0.0.0/0) matches everything else. ip route get 10.0.2.10 shows the decision for one address. Routes are written by hand (static), learned from the connected networks, or learned from a routing protocol like OSPF or BGP.

How a routing table works, step by step

Every packet a Linux machine sends or forwards goes through the same lookup. Take branchRouter, with a laptop network 10.0.1.0/24 on one side, two links to hqRouter (10.0.12.0/24, the primary, and 10.0.13.0/24, the backup), and two servers behind hqRouter, web at 10.0.2.10 and files at 10.0.2.11:

  1. Pick the table. The kernel walks its policy rules (ip rule) in order of preference. By default there are three: local (the machine's own addresses), main (what ip route shows) and default (empty). A rule can send some packets to another table, by incoming card, source address or firewall mark.
  2. Find every route that contains the destination. For 10.0.2.11, both 10.0.2.0/24 via 10.0.12.3 and a host route 10.0.2.11/32 via 10.0.13.3 match, and so does default (0.0.0.0/0).
  3. Longest prefix wins. The /32 is more specific than the /24, which beats the /0. Packets to the file server cross the backup link; packets to 10.0.2.10 still match only the /24 and take the primary.
  4. Among equal prefixes, the lowest metric wins. Two default routes with metrics 100 and 200 both sit in the table; the 200 is used only when the 100 is gone. Equal metrics with several next hops (nexthop via ... nexthop via ...) share the load (ECMP).
  5. Resolve the next hop. via 10.0.12.3 must be on a connected network. The kernel ARPs for it and builds the frame with that MAC; the IP destination stays 10.0.2.11.
  6. No match at all means the packet is dropped. A host gets Network unreachable; a router also sends ICMP net unreachable to the source.

Routes come from three places: the kernel adds one for each address's network (proto kernel scope link), people add static routes, and routing daemons such as FRR install what OSPF or BGP learn. When two sources offer the same prefix, the daemon chooses by administrative distance before anything reaches the kernel. Metric decides only between routes already in the kernel table.

What it looks like on a Linux machine

In the static routes lab branchRouter gets a /32 for the file server over the backup link, and ip route get shows the decision for each address:

$ ip route get 10.0.2.11
10.0.2.11 via 10.0.13.3 dev backupLink src 10.0.13.2 uid 0
    cache
$ ip route get 10.0.2.10
10.0.2.10 via 10.0.12.3 dev primaryLink src 10.0.12.2 uid 0
    cache

Two default routes added by hand with ip route add default via 10.0.12.3 metric 100 and the same via 10.0.13.3 with metric 200 both stay in the table:

$ ip route show default
default via 10.0.12.3 dev primaryLink metric 100
default via 10.0.13.3 dev backupLink metric 200

Extra tables and rules are how one Linux router keeps customers apart in the VRF-lite lab. Each customer's card looks up its own numbered table:

$ ip rule show
0:	from all lookup local
100:	from all iif officeA lookup 10
200:	from all iif officeB lookup 20
32766:	from all lookup main
32767:	from all lookup default
$ ip route get 192.168.50.10 from 10.0.1.10 iif officeA
192.168.50.10 from 10.0.1.10 via 10.0.2.10 dev serversA table 10
    cache iif officeA

ip route show table 10 lists one table; ip route show table all lists them all.

What goes wrong, and how you see it

No route back. branchRouter has 10.0.2.0/24 via 10.0.12.3, but hqRouter has nothing for 10.0.1.0/24. Pings leave and replies die at hqRouter. Every router on the path needs a route for both directions; the routing across hops lab shows each router knowing one step more.

A more specific route you forgot. An old /32 or a VPN's routes beat your new /24, so traffic goes the wrong way. ip route get <address> names the route actually used; reading ip route by eye misses it.

A next hop that is not on a connected network. ip route add 10.0.9.0/24 via 10.0.5.1 on a host without 10.0.5.0/24 fails with Error: Nexthop has invalid gateway.

A duplicate prefix. Adding a second route for the same prefix and metric in one table fails with RTNETLINK answers: File exists. Use another metric, ip route replace, or, for two customers with the same addresses, a second table.

A route that stays while the path is dead. A static route remains as long as its card is up, whether or not the next hop answers. Floating statics, tracking or a routing protocol fix it. In the routing across hops lab the middle link is cut, the routes stay, and traceroute shows where the answers stop.

Questions people ask

What is longest prefix match? Choosing the matching route with the most network bits: a /32 beats a /24, which beats the default /0. It lets one specific exception override a broad route. Try the boundaries with the subnet calculator.

What is the difference between metric and administrative distance? Distance ranks sources (connected, static, OSPF, BGP) for the same prefix inside a routing daemon. Metric ranks routes of one source, and in the Linux kernel it is the only tiebreak left.

How do I see the routing table on Linux? ip route for the main table, ip -6 route for IPv6, ip rule for the rules, ip route get for one address. route -n and netstat -rn are the older views. The bare machine lab builds a table from nothing.

See it on real machines

Learn it on real machines.

Every lab is a small network of real Linux machines in your browser. Start with a free account: Google or email, no card.

Start this lab, free