Labs / Recon / Recon: packets, banners and DNS

Lab 1 of 3 · Recon: packets, banners and DNS

curl: reading what a web server gives away

Fingerprint a booking server with curl alone: its software and version, its redirect, its robots.txt, and a backup file left in the open.

Start this lab 5 steps

The idea

Rafael has an authorised engagement against Zephyr Air, a regional airline, starting with its public booking server at 10.0.1.20. His whole first hour is reconnaissance: learn what the target is before touching how it breaks.

He opens no browser. A browser renders the page and throws the rest away; Rafael wants the rest. With curl alone he reads what the server volunteers to any client: the software and version in its headers, where it redirects, the paths it asks crawlers to skip, and a file someone left where it should not be.

A browser keeps the page; a tester keeps the envelope
A browser keeps the page; a tester keeps the envelope
The client makes the second request
The client makes the second request
A note, not a lock
A note, not a lock
No listing, and the file still comes
No listing, and the file still comes
Say less, serve less
Say less, serve less

Steps

  1. Ask the server to introduce itself

    Every HTTP reply comes in two parts: a block of headers, then the body (the page). The first line of the headers is the status line, a number that says how the request went. A browser shows you the body. The headers are where the server describes itself, and they are the first thing a tester reads.

    curl -I fetches the headers only. It sends a HEAD request, which means "send me the headers you would send for a GET, but not the body". So you read the server's own description of itself without pulling a single page. Point it at the site root and read what comes back.

    laptop:~# curl -sI http://10.0.1.20/
  2. Follow where it sends you

    The root answered 301 Moved Permanently with a Location header. A redirect is the server saying "not here, there", and the real application lives at there. A browser follows it without a word, so a user never notices; a tester has to follow it on purpose, because the thing worth mapping is wherever the chain ends.

    curl does not follow redirects unless you tell it to. -L makes it follow the Location header to the next response, and the next, until a page actually answers. Keep -I so you watch each hop as headers.

    laptop:~# curl -sIL http://10.0.1.20/
  3. Read what they ask crawlers to skip

    Almost every site serves a file at /robots.txt. It is a politeness note to search engines: a list of paths the owner would rather was not crawled and indexed. Google obeys it; an attacker reads it as a map.

    Think about what ends up in that list. An admin does not ask Google to skip the public home page; they ask it to skip the pages they would rather the world did not find: control panels, staging areas, backups. The file that is meant to hide those paths from a search index is the fastest way to learn they exist.

    laptop:~# curl -s http://10.0.1.20/robots.txt
  4. The file left in the open

    Nothing on the site links to /admin-dashboard/ or /backups/, but robots.txt named both. A path no page links to is not hidden, only unlinked: the server answers a request for it like any other.

    Backups under the web root are a classic, costly slip: an editor's save file, a config.bak a deploy script copied and forgot. The web server has no idea a file is sensitive; it serves it like any other.

    Start with the staff page, then the backups folder itself. A folder that will not list its files can still serve any file you name, and backup names are predictable: config.bak, backup.zip and db.sql sit near the top of every Content discovery wordlist.

    laptop:~# curl -s http://10.0.1.20/admin-dashboard/
    laptop:~# curl -sI http://10.0.1.20/backups/
    laptop:~# curl -s http://10.0.1.20/backups/config.bak
  5. Shut the server up

    Now swap seats. You run Zephyr Air's servers, and Rafael's report lists four gifts: the version banner, the framework header, a robots.txt that advertises sensitive paths, and a backup under the web root. The fix is to stop volunteering what recon reads.

    Turn off server_tokens so nginx stops printing its version, drop the X-Powered-By header, and get the backup off the web server entirely. A backup never belongs under the web root. Then look at the server's own access log and ask the harder question: could you even have caught Rafael doing this?

    web:~# nginx -s reload
    web:~# rm -f /srv/www/backups/config.bak
    laptop:~# curl -sI http://10.0.1.20/
    laptop:~# curl -s http://10.0.1.20/backups/config.bak
    web:~# tail -n 20 /var/log/nginx/access.log

Run these steps on real machines

Keep reading

Learn it on real machines.

Every lab is a small network of real Linux machines in your browser. Sign in with Google or email to start.

Start this lab