Labs / Defensive Tools / Block and contain: the host, the wire and the data

Lab 2 of 3 · Block and contain: the host, the wire and the data

Inline IPS: the gateway drops, not only watches

Put Suricata on the forwarding path through NFQUEUE, then block a flagged request while the floor keeps browsing.

Start this lab 5 steps

The idea

Femi looks after the network at Marigold Gardens, a care home. This morning the security feed flagged a path that scanning kits probe for, /server-status, and the lobby kiosk keeps asking the resident portal for it. A sensor on a mirror can only tell Femi about it afterwards. Femi wants it stopped, without cutting the care floor off from the portal.

So the same Suricata moves onto the path, into the gateway that forwards every packet between the floor and the portal. There it becomes an IPS: it holds each packet in a netfilter queue until it says accept or drop. You start it watching, change one word in a rule from alert to drop, and decide what happens if it dies.

Beside the path alerts; on the path can drop
Beside the path alerts; on the path can drop
Queued is not blocked
Queued is not blocked
One word decides the verdict
One word decides the verdict
One verdict per packet
One verdict per packet
A dead reader, two outcomes
A dead reader, two outcomes

Steps

  1. On the path, not beside it

    A sensor on a mirror port sits off to the side, reading copies. It can raise an alert, but the real packet has already gone on.

    The gateway is different. It forwards every packet between the care floor (10.0.1.0/24) and the portal (10.0.2.10), so whatever the gateway decides about a packet is final.

    That is the difference between an intrusion detection system (IDS) and an intrusion prevention system (IPS). The engine can be the same. What changes is where it sits. Watch a request from the care desk reach the portal, and trace the path it takes.

    careDesk:~# traceroute -n 10.0.2.10
    careDesk:~# curl -sS http://10.0.2.10/
  2. Into the queue: every packet waits for a verdict

    On the mirror, Suricata read copies off a network card. Inline, it reads a netfilter queue instead. One iptables rule in the FORWARD chain hands every forwarded packet to queue 0 (NFQUEUE). The kernel holds each packet there until Suricata answers accept or drop, then does what Suricata said.

    That answer is the verdict.

    Femi starts carefully, with one rule whose action is alert: the queue works, but nothing is blocked yet. The iptables rule also gets --queue-bypass, a safety valve. If nothing is reading the queue, the kernel lets packets through instead of holding them. The last step tests that valve.

    gateway:~# cat /etc/suricata/rules/local.rules
    gateway:~# suricata -c /etc/suricata/netlab.yaml -q 0 -D -l /var/log/suricata
    gateway:~# grep -E 'rules successfully loaded|Engine started' /var/log/suricata/suricata.log
    gateway:~# iptables -I FORWARD 1 -j NFQUEUE --queue-num 0 --queue-bypass
    careDesk:~# curl -sS http://10.0.2.10/
  3. Alert watches, drop blocks

    Suricata is inline now, but the rule's action is still alert. An alert rule writes a log line and accepts the packet, even on the path. The first word of the rule, the action, decides what happens to a packet that matches.

    First see the kiosk's request get through under alert. Then Femi changes alert to drop, reloads the rules live, and the kiosk tries again. The pattern, the port and the direction stay the same. Only the action changes.

    lobbyKiosk:~# curl -sS http://10.0.2.10/server-status
    gateway:~# jq -c 'select(.event_type=="alert") | {action: .alert.action, url: .http.url}' /var/log/suricata/eve.json
    lobbyKiosk:~# curl -sS -m6 http://10.0.2.10/server-status
    gateway:~# cat /var/log/suricata/fast.log
    gateway:~# jq -c 'select(.event_type=="alert") | {action: .alert.action}' /var/log/suricata/eve.json
  4. Only the match is stopped

    Femi's real worry is the care floor. Nurses use the portal all day, and they must not notice any of this.

    A drop verdict applies to the matching packet only. Every other packet still gets its own verdict, and with no matching rule that verdict is accept.

    This is also why a careless rule is dangerous inline. A pattern that matches too much drops real traffic and takes the floor down. On a mirror, too broad a rule makes noise. On the path, it makes an outage.

    careDesk:~# curl -sS http://10.0.2.10/
    lobbyKiosk:~# curl -sS -m6 http://10.0.2.10/server-status
  5. Fail open, or fail closed

    A sensor on a mirror can crash and nobody on the floor notices. An inline one forces a question: what happens to the traffic when Suricata dies?

    The kernel has two choices. It can accept packets when nothing reads the queue, which is fail open: the network stays up, but nothing inspects it. Or it can drop them, which is fail closed: nothing gets past unchecked, because nothing gets past at all.

    --queue-bypass picks between the two. A loss meter pings the portal from the care desk while the lab stops Suricata and leaves the NFQUEUE rule in place.

    careDesk:~# ping -i 0.2 -c 80 -W 1 10.0.2.10
    gateway:~# iptables -L FORWARD 1 -n

Run these steps on real machines

Learn it on real machines.

Every lab is a small network of real Linux machines in your browser. Sign in with Google or email to start.

Start this lab