Labs / Tunnels & VPNs / Private links

Lab 1 of 4 · Private links

GRE tunnels: send a private network across someone else's

Build a GRE tunnel across a carrier, read the wrapped packets, and collapse it once.

Start this lab 7 steps

The idea

Noor's two ferry offices use private addresses, and the carrier between them won't carry those. Wrap each packet inside a public one, then find out what the wrapper costs.

Public crosses, private stops
Public crosses, private stops
Two packets, one on the wire
Two packets, one on the wire
A route sends traffic into the tunnel
A route sends traffic into the tunnel
One journey, two traces
One journey, two traces
Three pings around 1476
Three pings around 1476
A hello with TTL 1 crosses two routers
A hello with TTL 1 crosses two routers
A tunnel can't be its own route
A tunnel can't be its own route

Steps

  1. Ask the island for its bookings

    Noor looks after the computers at a small ferry company. The ticket desk at the harbour needs today's bookings, which live on a server at the island office, and the only thing between the two offices is a carrier, the company that runs the cable under the sea.

    Both offices use private addresses, so bookingServer is 10.40.7.20. Each office Router also has one public IP address on its line to the carrier: 198.51.100.2 at the harbour, 192.0.2.2 on the island.

    Ask for the page from ticketDesk. Then ask carrierWest, the carrier's router next to the harbour, where it would send that address, and ping the island's public address from harbourRouter.

    ticketDesk:~# curl -sS -m 3 http://10.40.7.20/
    carrierWest:~# ip route get 10.40.7.20
    harbourRouter:~# ping -c 1 -W 1 192.0.2.2
  2. Wrap a packet in a packet

    The carrier delivers anything addressed to a public address, so give it one. At the harbour, put the whole private packet inside a new packet addressed to the island's router. At the island, take it out and send it on. The carrier reads the outer addresses and never looks further.

    That's a tunnel, and the name for wrapping one packet in another is encapsulation. The idea wasn't invented to hide private addresses. In 1994 big networks still ran several protocols side by side, such as Novell's IPX and Apple's AppleTalk, and every pair needed its own way of carrying one over the other. RFC 1701 offered one general wrapper instead, Generic Routing Encapsulation (GRE), which doesn't care what it carries. RFC 2784 removed the optional fields in 2000, and the header in use today is 4 bytes.

    A GRE packet rides directly on IP, the way TCP and UDP do. It has its own protocol number, 47, where TCP has 6 and UDP has 17, and it has no ports.

    A router gets a tunnel as a virtual network card, a card with no cable. Whatever it sends through that card comes out wrapped, through the real one.

    A card needs an address, and a tunnel counts as a small network with two machines on it, so the two ends get 10.40.99.1 and 10.40.99.2. Make the card on harbourRouter, give it the first address, and ping the second. The lab records what leaves on the carrier line.

    harbourRouter:~# ip -echo link add toIsland up type gre local 198.51.100.2 remote 192.0.2.2 ttl 64
    harbourRouter:~# ip -echo addr add 10.40.99.1/30 dev toIsland
    harbourRouter:~# ping -c 1 -W 1 10.40.99.2
  3. Open the far end

    A tunnel takes two ends. islandRouter needs the same card with local and remote swapped, because on the way back the outer packet travels the other way.

    Make that card, then ping across it. In the first step the ping between public addresses came back with ttl=62, one less for each carrier router. Watch that number this time.

    With the tunnel working, try the bookings again from ticketDesk, and find out what harbourRouter does with that address.

    islandRouter:~# ip -echo link add toHarbour up type gre local 192.0.2.2 remote 198.51.100.2 ttl 64
    harbourRouter:~# ping -c 2 -W 1 10.40.99.2
    ticketDesk:~# curl -sS -m 3 http://10.40.7.20/
    harbourRouter:~# ip route get 10.40.7.20
    harbourRouter:~# ip -echo route add 10.40.7.0/24 via 10.40.99.2
    ticketDesk:~# curl -sS -m 3 http://10.40.7.20/
  4. Read the carrier's line

    The desk now sees a straight road to the island. The carrier sees something else, and Noor has a question about that, because bookingServer has a second page, /manifest, with the names of the passengers on the morning sailing.

    Compare the two views first with traceroute, from the desk to the server and then from harbourRouter to the island's public address. Each line of its output is a router the packet passed through.

    Then fetch the manifest while the lab records on carrierWest, on the carrier's own line. On a real network, you'd never get to log in to a carrier's router, and this is the view of anyone who can, or who can tap a link along the way.

    ticketDesk:~# traceroute -n 10.40.7.20
    harbourRouter:~# traceroute -n 192.0.2.2
    ticketDesk:~# curl -sS -m 3 http://10.40.7.20/manifest
  5. Pay for the wrapper

    The carrier's lines take packets up to 1500 bytes, their MTU. A wrapped packet has two more headers: 20 bytes for the outer IP header and 4 for GRE. That's 24 bytes the passenger can't use, so the tunnel card took an MTU of 1476 by itself. You saw it in the output when you made the card.

    Your desk doesn't know that. Its own cable takes 1500, so it sends full-size packets, and a web page is mostly full-size packets. The question is what harbourRouter does with a 1500-byte packet it can't wrap.

    Test the edge with ping. -M do sets the don't fragment flag, which tells routers to refuse the packet instead of cutting it up. -s sets the bytes of data, and ping adds an 8-byte header of its own, on top of the 20-byte IP header. So -s 1472 makes a packet of exactly 1500, which fits every cable on the path.

    Send that one first, then two more around the edge.

    ticketDesk:~# ping -c 1 -W 1 -M do -s 1472 10.40.7.20
    ticketDesk:~# ping -c 1 -W 1 -M do -s 1449 10.40.7.20
    ticketDesk:~# ping -c 1 -W 1 -M do -s 1448 10.40.7.20
    ticketDesk:~# ip route get 10.40.7.20
  6. Let the routers talk through it

    Noor's tunnel has two routes typed by hand, and the company is about to open a second network on the island. Each new network means logging in at both ends, and each mistake means a silent failure. Routers usually learn this for themselves by talking, with a routing protocol such as OSPF.

    OSPF routers find each other with hellos, short "I'm here" messages sent every few seconds, and those can't cross a carrier by themselves, for two reasons. They go to a multicast address, 224.0.0.5, which means "every OSPF router on this wire", and they leave with a TTL of 1, so they stop at the next router.

    This is where GRE earns its keep. Wrapped, those messages are ordinary public packets to the carrier, and to the routers the tunnel is one wire with two ends. That's a big part of why GRE is still in use, 30 years on.

    harbourRouter and islandRouter already run OSPF on their other cards, with no neighbour to talk to. Turn it on for each tunnel card, in area 0, the one group of routers this network has. Then read a hello as it crosses the carrier, and check that the two have become neighbours.

    There's a catch in those other cards. islandRouter tells its neighbours about every network it has OSPF on, and that includes 192.0.2.0/24, its carrier line. So once the two are talking, harbourRouter will hear of a route to that block through the tunnel, and the most specific route always wins: a /24 beats the default route that has been carrying the tunnel so far.

    harbourRouter:~# vtysh -c 'configure terminal' -c 'interface toIsland' -c 'ip ospf area 0'
    islandRouter:~# vtysh -c 'configure terminal' -c 'interface toHarbour' -c 'ip ospf area 0'
    harbourRouter:~# vtysh -c 'show ip ospf neighbor'
    harbourRouter:~# ip route get 192.0.2.2
    harbourRouter:~# ip -s link show toIsland
    harbourRouter:~# ip route get 192.0.2.2
  7. Keep the far end outside

    There's one rule for routing over a tunnel: the far end's address must always be reached around the tunnel, never through it.

    Real networks keep that rule in one of three ways. They pin the far end with a static route to that one address, which is the fix in Cisco's note, because a /32 is as specific as a route gets. They filter the far end out of what the tunnel's routers accept. Or they keep the carrier lines out of the routing protocol altogether, so that nobody ever hears about them, and that's the cleanest here: the outer network and the inner one are separate, and neither should be talking about the other.

    Turn OSPF off for the carrier card at each end, and check that 192.0.2.2 stays around the tunnel. Then take out the routes you typed, and see whether the bookings still load.

    islandRouter:~# vtysh -c 'configure terminal' -c 'interface eastLine' -c 'no ip ospf area 0'
    harbourRouter:~# vtysh -c 'configure terminal' -c 'interface westLine' -c 'no ip ospf area 0'
    harbourRouter:~# ip route get 192.0.2.2
    harbourRouter:~# ip route del 10.40.7.0/24 via 10.40.99.2
    harbourRouter:~# ip route
    ticketDesk:~# curl -sS -m 3 http://10.40.7.20/

Run these steps on real machines

Keep reading

Learn it on real machines.

Every lab is a small network of real Linux machines in your browser. Sign in with Google or email to start.

Start this lab