Labs / Night Watch / Forensics, incident response & log correlation

Lab 3 of 3 · Forensics, incident response & log correlation

Incident response: working a live compromise by the runbook

Find and clean up a live compromise step by step, following the runbook.

Open this lab Pro 6 steps

The idea

Incident response works a live compromise by a runbook, in order: identify it, contain it without destroying evidence, eradicate the implant and whatever would bring it back, recover the service, and record what you learned. webServer on the office LAN has been backdoored to call c2 on the internet; you work on webServer and the router, and the responder checks the web page.

Two parts to the implant, two places to cut
Two parts to the implant, two places to cut

Steps

  1. Identification: find the compromise

    You keep a baseline of each server, so the abnormal stands out. This one should answer its web page and listen on one port only. The implant opens no port, it reaches out, so a check of open ports looks clean; the process list is where it shows: a .sysupdate script you do not recognise.

    webServer:~# ss -tlnp
    webServer:~# ps -o pid,args
    webServer:~# cat /usr/local/bin/.sysupdate
  2. Find the channel: the beacon in the router's connections

    Confirm it from a second, independent view: the network, because an attacker on the server cannot wipe what the router saw. The router tracks every connection it forwards, so the beacon's repeated calls to the C2 show up in its connection table.

    router:~# conntrack -L -d 10.0.2.10 -p tcp

Pro

4 more steps in the app

  1. Containment: cut the beacon, keep the evidence
  2. Eradication: remove the implant and its persistence
  3. Recovery: restore service and confirm it is quiet
  4. Lessons learned: write it down and hunt wider
Get Pro

Learn it on real machines.

Every lab is a small network of real Linux machines in your browser. Start with a free account: Google or email, no card.

Open this lab