Labs / Protocols / Ports, TCP & UDP

Lab 2 of 3 · Ports, TCP & UDP

TCP: how a file survives a bad line

A radio line loses one packet in five, and the file still arrives whole.

Start this lab 7 steps

The idea

Rosa keeps a lighthouse on a rock at sea. Its only link to shore is a radio line that fails in every storm, and the lamp's software still has to arrive whole.

IP keeps no copy
IP keeps no copy
TCP runs only at the two ends
TCP runs only at the two ends
The three-way handshake
The three-way handshake
Two numbers in every reply
Two numbers in every reply
The same ack, again and again
The same ack, again and again
One loss, two costs
One loss, two costs
The depot waits longer after each try
The depot waits longer after each try

Steps

  1. Test the radio line

    Rosa's lighthouse, 10.0.1.10, reaches the depot on shore, 10.0.2.10, over the radio line and a Router called mast. A storm is on the line now.

    A network promises very little. The Internet Protocol (IP) passes each packet from router to router and keeps no copy of it. If a line loses the packet, IP does nothing about it and tells nobody.

    Send 30 pings from the lighthouse to see how much gets through.

    lighthouse:~# ping -c 30 -i 0.2 10.0.2.10
  2. Download the lamp update through the storm

    Rosa needs the lamp update, a file of 131072 bytes. One packet holds about 1400 bytes, so the depot sends the file in about 90 packets. At one packet in five, the storm will take about 18 of them.

    The file size alone can't prove the download worked, because a damaged file can have the right size. So Rosa also downloads the depot's hash of the file. A hash is a short fingerprint of a file: change one byte and the hash changes.

    Download both files, then compare them with sha256sum -c.

    lighthouse:~# curl -s -w '%{size_download} bytes in %{time_total} seconds\n' -O http://10.0.2.10:8000/lamp-update.bin -O http://10.0.2.10:8000/lamp-update.sha256
    lighthouse:~# sha256sum -c lamp-update.sha256
  3. Watch a connection open and close

    IP already delivers packets, so why does TCP set anything up first? TCP's whole job rests on numbering bytes, and two machines have to agree on the numbers before they can use them.

    Each machine picks its own starting number. An attacker can't guess it, and a stray packet from an old connection is unlikely to fit it.

    Each one must tell the other that number and hear that it arrived. On a line that loses packets, neither machine can assume it did.

    Ask the depot for version.txt, a tiny file, while the lab records every packet in a capture with tcpdump. The line stays calm for this step. Then list the depot's connections with ss -tan, to see what a closed connection leaves behind.

    lighthouse:~# curl -s http://10.0.2.10:8000/version.txt
    depot:~# ss -tan
  4. Count bytes, not packets

    TCP numbers every byte it sends. A program hands TCP one long stream of bytes, and TCP cuts it into packets as it sees fit. A count of packets would mean nothing.

    A packet's seq says which bytes it holds. ack, short for acknowledge, is the number the other machine sends back to say how much has arrived.

    Every ack also carries win, for a different problem. The depot can send faster than a program on the lighthouse reads, and those bytes need somewhere to wait. So the lighthouse keeps telling the depot how much room it has left.

    Run the same fetch again and read seq and ack. Then start a download that reads 8 KB a second, on a line that carries over 100 KB a second. The lab records only the lighthouse's replies.

    lighthouse:~# curl -s http://10.0.2.10:8000/version.txt
    lighthouse:~# curl -s --limit-rate 8K -m 7 -o /dev/null -w '%{size_download} bytes in %{time_total} seconds\n' http://10.0.2.10:8000/sea-charts.bin
  5. Find the packet that went missing

    A router that loses a packet on a line or in a full queue tells nobody. The two ends have to work it out from the numbers. ack can say only one thing: the next byte wanted.

    Suppose bytes 1000 to 1999 go missing while later bytes arrive. The lighthouse can't say "I have 2000 onward" with ack alone, so it keeps saying "I still want byte 1000". Those repeats are the signal.

    The line now loses one packet of a small file, the tide table, so the capture stays short. In rows with a seq range carry the file from the depot. Out rows are the lighthouse's replies.

    lighthouse:~# curl -s -w '%{size_download} bytes in %{time_total} seconds\n' -O http://10.0.2.10:8000/tide-table.txt
  6. Count the resends in a storm

    A lost packet costs TCP twice. The first cost is the resend, and the depot's counters show that one. The second cost is speed, and it comes from a bad day in the internet's history.

    In October 1986, transfers between two sites in Berkeley fell from 32 Kbit/s to 40 bit/s. Machines kept resending into full lines, and the resends filled those lines further. The fix went into TCP: when packets go missing, slow down.

    The storm is back on the line. Read the depot's counters, download the lamp update again, and read them once more.

    depot:~# nstat -az TcpOutSegs TcpRetransSegs
    lighthouse:~# curl -s -w '%{size_download} bytes in %{time_total} seconds\n' -O http://10.0.2.10:8000/lamp-update.bin
    depot:~# nstat -az TcpOutSegs TcpRetransSegs
  7. Cut the line during a download

    A connection feels like a wire between two machines, but a plain router on the way keeps no record of it. The designers of the internet chose this on purpose. They kept a connection's record in the two machines, so a router that dies need not take a connection with it.

    This step breaks the line on purpose, and the lab brings it back. Start the sea charts, a 13-second download, and the lab cuts the line for 8 seconds. The whole step takes about 30 seconds.

    During the cut, the depot prints its list of connections.

    lighthouse:~# curl -s -w '%{size_download} bytes in %{time_total} seconds\n' -O http://10.0.2.10:8000/sea-charts.bin -O http://10.0.2.10:8000/sea-charts.sha256
    depot:~# ss -tn
    lighthouse:~# sha256sum -c sea-charts.sha256

Run these steps on real machines

Keep reading

Learn it on real machines.

Every lab is a small network of real Linux machines in your browser. Sign in with Google or email to start.

Start this lab