GRE tunnel

GRE (RFC 2784) puts a 4-byte header and a new IP header (protocol 47) in front of the packet, so a network that cannot route the inner addresses, such as the internet with private ones, carries it between the two tunnel endpoints. It costs 24 bytes, so the tunnel's MTU is 1476. It carries multicast and routing protocols, but it neither encrypts nor authenticates, and it keeps no state: without keepalives or a routing protocol over it, a tunnel to a dead peer stays up. Across the internet it is run inside IPsec. Linux: ip tunnel add … mode gre; Cisco: interface Tunnel0.

How a GRE tunnel works, step by step

Two sites, 10.0.1.0/24 behind routerA and 10.0.2.0/24 behind routerB. The routers' outside addresses are 10.0.11.2 and 10.0.12.2, and the network between them, standing in for the internet, routes only those. Each router gets a tunnel card, and the two cards share a small link network, 172.16.0.0/30. Follow alice (10.0.1.10) pinging the file server (10.0.2.10):

1. routerA looks up the destination. Its route 10.0.2.0/24 via 172.16.0.2 points out of the tunnel card, toSiteB. The tunnel is an ordinary link to the routing table.
2. The tunnel card encapsulates. In front of alice's 84-byte packet it puts a GRE header (RFC 2784: 4 bytes, flags and version 0, then the protocol type of the payload, 0x0800 for IPv4, the same EtherType numbers Ethernet uses) and a new outer IP header: source 10.0.11.2, destination 10.0.12.2, protocol 47. The packet is now 108 bytes.
3. The internet routes the outer header only. It never reads the inner addresses and never lowers the inner TTL, so the whole crossing counts as one hop.
4. routerB decapsulates. A packet with protocol 47 from 10.0.11.2 matches its tunnel toSiteA (remote 10.0.11.2). It strips both headers and routes alice's packet onto its LAN as if it had arrived on a cable.
5. The reply takes the mirror path, wrapped by routerB with the addresses swapped.

Optional fields (RFC 2890) add a checksum, a 4-byte key to tell several tunnels between the same two addresses apart, and a sequence number. Because the payload type is an EtherType, GRE carries more than IPv4: IPv6, MPLS, or whole Ethernet frames (gretap). It sets up no session and sends no hellos of its own.

What it looks like on a Linux machine

The GRE tunnels lab builds both ends with iproute2. On routerA:

ip tunnel add toSiteB mode gre local 10.0.11.2 remote 10.0.12.2 ttl 64
ip addr add 172.16.0.1/30 dev toSiteB
ip link set toSiteB up
ip route add 10.0.2.0/24 via 172.16.0.2

routerB runs the mirror, local 10.0.12.2 remote 10.0.11.2. The card reports its kind and size:

$ ip -d link show toSiteB
11: toSiteB@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1476 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/gre 10.0.11.2 peer 10.0.12.2 promiscuity 0 allmulti 0 minmtu 68 maxmtu 65511
    gre remote 10.0.12.2 local 10.0.11.2 ttl 64 ...

On the router in the middle, tcpdump -nn -v proto gre shows a packet inside a packet:

IP (tos 0x0, ttl 64, id 5875, offset 0, flags [DF], proto GRE (47), length 108)
    10.0.11.2 > 10.0.12.2: GREv0, Flags [none], length 88
	IP (tos 0x0, ttl 63, id 25698, offset 0, flags [DF], proto ICMP (1), length 84)
    10.0.1.10 > 10.0.2.10: ICMP echo request, id 73, seq 1, length 64

Alice's traceroute lists 10.0.1.2, then 172.16.0.2, then 10.0.2.10: the middle router never appears.

What goes wrong, and how you see it

One end points at the wrong address. GRE has no handshake, so both cards say UP even when routerB's remote is a typo. Packets arrive at routerB, its answers go nowhere, and alice sees 100% packet loss. ip tunnel show toSiteA on routerB prints toSiteA: gre/ip remote 10.0.11.9 local 10.0.12.2 ttl 64; fix it with ip tunnel change toSiteA remote 10.0.11.2. tcpdump proto gre in the middle shows traffic in one direction only.

The path dies, the tunnel stays up. Cut the link to routerB and routerA still shows toSiteB up with 10.0.2.0/24 via 172.16.0.2 dev toSiteB, so nothing fails over. Run OSPF or BGP over the tunnel, or keepalives, so the route goes away when the far end does.

Big packets vanish. The tunnel's MTU is 1476. A ping with -M do -s 1449 gets Frag needed and DF set (mtu = 1476) from 10.0.1.2; if those ICMP messages are filtered, big downloads hang while small ones work. See MTU and MSS clamping.

A firewall or NAT in the path. GRE is protocol 47, not TCP or UDP. A filter that allows only ports drops it, and many NAT devices cannot translate it because there is no port to tell flows apart.

Anyone on the path reads it. The lab's capture prints alice,61000 from payroll.csv in plain text.

Questions people ask

Is GRE a VPN? It is a tunnel without security: no encryption, no authentication. Across the internet it runs inside IPsec (GRE over IPsec), where GRE carries multicast and routing protocols and IPsec protects them. The IPsec site-to-site lab shows the encrypted kind.

GRE vs IPIP vs VXLAN? IPIP (protocol 4) carries only IPv4 and costs 20 bytes. GRE costs 24 and carries any protocol. VXLAN carries Ethernet frames in UDP port 4789 for 50 bytes, which load-balances better in data centre fabrics.

Does GRE work over NAT? Usually not well. Use GRE inside IPsec with NAT traversal, or a UDP-based tunnel.

See it on real machines