The idea
An auditor says patient records cross the carrier in plain text, and two clinics' GRE tunnel doesn't stop that. Encrypt it with IPsec, then see what fails and what still shows.
Steps
-
Read what the carrier reads
Rosa runs the computers at two clinics in the same valley. Riverside has the reception desks and Hillside has the patient records, and for years a GRE tunnel across the carrier between them has made the two feel like one building.
This morning an auditor left one line in her report: patient records cross the carrier in plain text. Rosa doesn't believe it, because a tunnel sounds private. But GRE only wraps each packet in a new one so that a network knowing nothing about the clinics can carry it, and a wrapper isn't a lock.
Stand where the carrier stands. The lab records on
carrierwithtcpdump -A, which prints each packet's bytes as text, whilereceptionPcfetches a patient's record fromrecordsServer.receptionPc:~# curl -sS -m 3 http://10.60.7.20/patient/4471
-
Agree on keys in the open
Encryption is the solved half of this problem. The unsolved half is that both routers need the same secret key, and the only path between them runs through the carrier you've stopped trusting.
Early encrypted links used couriers, who carried keys on punched cards and paper tape. Anyone who copied a key could open every recording made under it. In 1976 Whitfield Diffie and Martin Hellman published a way around the courier: each side picks a private number, sends a public value made from it, and combines the other side's public value with its own private number. Both ends arrive at the same result, and someone who recorded both public values can't work it out.
IPsec does that in a protocol called IKE, the Internet Key Exchange, and it takes two exchanges.
IKE_SA_INITagrees on a cipher and does the Diffie-Hellman swap in the open. That proves nothing about who's at the other end, since a stranger in the middle could do the same swap, soIKE_AUTHruns under the new key and each side proves who it is. Here the proof is a secret both clinics were given in advance, called a pre-shared key. It proves who's talking and encrypts nothing, because a key that encrypted the traffic would open every saved recording on the day it leaked. Then IKE derives a second set of keys for the traffic itself.The IPsec software on both routers is strongSwan, and
hillRouteralready has its half of the config.riverRouter's file is waiting for you.local_tsandremote_tsare its traffic selectors: the list of what this tunnel protects, by network.start_action = trapmeans no packet crosses when you load it. The router waits for the first packet that matches.receptionPc:~# ping -c 3 -i 0.3 -W 1 10.60.7.20 riverRouter:~# grep IKE_ /var/log/charon.log riverRouter:~# swanctl --list-sas
-
Read what's left on the wire
IKE agreed on keys and stepped aside. The traffic itself now travels in ESP, the Encapsulating Security Payload. ESP is its own IP protocol, number 50, sitting beside TCP (6) and UDP (17), and it has no ports.
Three things happen to each packet for Hillside as it leaves
riverRouter. The whole original packet, addresses included, becomes the payload. AES-GCM encrypts that and adds a 16-byte tag, so a packet changed on the way fails its check at the far end and is thrown away. Then a new outer IP header goes on the front, from router to router, followed by a short ESP header holding two numbers. The SPI, the Security Parameter Index, names the keys this packet uses. The sequence number goes up by one per packet, and with replay protection on, as here, the far end refuses any number it has seen, so nobody can replay a recorded packet.The Linux kernel does this work, not strongSwan. strongSwan hands the kernel two lists: policies, which say what to protect, and states, which hold the keys.
That raises a question, because
riverRouter's routing table hasn't changed, and it still sends Hillside's traffic intotoHill, the virtual card GRE gives the router. So fetch the record again with the carrier recording, read both lists, and ask the router which way it would send that packet.receptionPc:~# curl -sS -m 3 http://10.60.7.20/patient/4471 riverRouter:~# ip xfrm state riverRouter:~# ip route get 10.60.7.20 from 10.60.1.10 iif riverClinic riverRouter:~# ip xfrm policy list dir out
-
Get the key wrong
Tunnels fail. Someone mistypes a secret at a change window, a certificate expires at 3 a.m., a Firewall upgrade drops a port. So the question that matters more than the cipher is: when this tunnel goes down, what happens to the records?
There are two designs, with two names you'll see on security exams. A system that fails open keeps working unprotected: the doctors notice nothing and neither does anyone else. A system that fails closed stops: the doctors phone you, and nothing leaks.
Rosa has a reason to worry here. The GRE route is still in the routing table, and an hour ago that route carried this same record past the carrier in plain text. If the policy steps aside when the keys are gone, the routing table will send the record the old way.
So make the tunnel fail. Change the pre-shared key on
riverRouteronly, close the running tunnel so the next packet needs fresh keys, and fetch the record with the carrier recording.riverRouter:~# swanctl --terminate --ike hill receptionPc:~# curl -sS -m 3 http://10.60.7.20/patient/4471 riverRouter:~# grep AUTHENTICATION_FAILED /var/log/charon.log hillRouter:~# grep 'shared key' /var/log/charon.log riverRouter:~# ip xfrm policy list dir out receptionPc:~# curl -sS -m 3 http://10.60.7.20/patient/4471
-
Bring the van through NAT
Hillside runs a clinic in a van, with a tablet in the back and a 4G modem on the roof, and the tablet needs the same records. Rosa has put
vanRouterin it, running strongSwan, and toldhillRouterto expect it at100.64.9.2, the address on its card.That address is the problem. 4G networks are short of public addresses, so they hand out shared ones from
100.64.0.0/10, a range set aside for carrier NAT, and put NAT at the edge, swapping each packet's source for a public address as it leaves.IPsec was designed for an internet where a machine's address stayed the same on the whole journey, and NAT broke it. Part of the break is that IKE config names a peer by address. The harder part is that NAT tells its customers apart by port number, and ESP has no ports. A TCP packet shows its ports to anyone, but ESP's only useful readable fields are an SPI and a counter. Home routers sold an "IPsec passthrough" feature that tracked one ESP flow by address, and it fell over as soon as two people behind the same router used the same VPN gateway.
The fix, called NAT traversal, came in three parts. In
IKE_SA_INIT, each end sends a hash of the address and port it thinks it's using, and the other end compares that with what arrived. A mismatch means something rewrote the packet. From then on, both ends wrap each ESP packet in a UDP header, port 4500, which gives NAT the ports it needs. And the end behind NAT sends a small keepalive every 20 seconds, so NAT doesn't forget the mapping.Start the van's tunnel and see how far it gets.
vanRouter:~# swanctl --initiate --child records hillRouter:~# grep 'no IKE config' /var/log/charon.log vanRouter:~# swanctl --initiate --child records vanTablet:~# curl -sS -m 3 http://10.60.7.20/patient/4471 vanRouter:~# swanctl --list-sas
-
A network nobody listed
Rosa's report for the auditor is nearly written. Then a nurse at Riverside mentions how handy the new scans are.
Hillside bought an X-ray machine last month, and
scannersits on its own network,imaging,10.60.8.0/24. Nobody touched the tunnel config for it. OSPF saw the new network and toldriverRouterabout it through GRE, which is what a routing protocol is for, and within seconds the scans were reachable from every reception desk.Fetch a scan with the carrier recording, then look at what the router knows and what the tunnel covers.
receptionPc:~# curl -sS -m 3 http://10.60.8.30/ riverRouter:~# ip route show 10.60.8.0/24 riverRouter:~# swanctl --list-pols
-
Encrypt the tunnel itself
A better design stops listing networks and protects the one thing they all cross. From outside, everything between the clinics is one flow: GRE, IP protocol 47, from
198.51.100.2to203.0.113.2. Put that flow in the selectors, and every network OSPF ever learns is inside it from the moment it appears.It also solves a problem you haven't hit yet. OSPF hellos go to a Multicast address, IPsec policies are built for traffic to one address, and IPsec gives OSPF no card to run on, so routing protocols never ran well over plain IPsec. Inside GRE, a multicast hello is an ordinary packet between two routers. That's why GRE over IPsec became a standard design: GRE gives you something a routing protocol can use, and IPsec makes it private.
It changes the mode too. Tunnel mode, which you've used until now, wraps the whole packet and adds a new outer IP header. GRE already has an outer header, with the right addresses in it, so a second one would be 20 wasted bytes on every packet. Transport mode keeps the packet's own header and encrypts everything after it.
hillRouterhas its half of the change. InriverRouter's config, therecordschild becomes one namedgre. Its selectors become the two routers' public addresses with[gre]on the end, which means that protocol only, andmode = transportpicks the mode.There's a cost to check afterwards. Each card has a largest packet it carries, its MTU, and
toHill's is 1476 bytes, because GRE's wrapper takes 24 of the usual 1500.riverRouter:~# swanctl --terminate --ike hill receptionPc:~# curl -sS -m 3 http://10.60.8.30/ riverRouter:~# swanctl --list-sas riverRouter:~# vtysh -c 'show ip ospf neighbor' receptionPc:~# ping -c 1 -W 1 -M do -s 1432 10.60.7.20
Run these steps on real machines
Keep reading
- strongSwan's introduction to IPsec: ESP, IKEv2 and every message, with diagrams article
- strongSwan on NAT traversal: why ESP needs UDP, and what passthrough can't do article
- swanctl.conf: every setting the lab's config files use, including start_action and encap article
- NetworkLessons on encrypting a GRE tunnel with IPsec, the same design on Cisco routers article