Commands / Ports and connections

Ports and connections

curl

Also wget, http client

Sends an HTTP or HTTPS request and prints the answer: the page, the headers, or the whole conversation.

The end-to-end test for the web: name lookup, TCP connection, TLS, HTTP request and response in one command. -v prints each step (* Trying…, * Connected, > request, < response), so where it stops is where the trouble is. Its exit code names the failing step: 6 could not resolve the name, 7 could not connect, 28 timed out, 35/60 TLS failed.

curl command usage

curl [-s [-S]] [-i | -I] [-v] [-L] [-f] [-o FILE | -O] [-w FORMAT] URL
curl [-X METHOD] [-H HEADER]... [-d DATA] [-u USER:PASSWORD] URL
curl [-b JAR] [-c JAR] [-A AGENT] [-e REFERER] [--resolve NAME:PORT:ADDRESS] URL
curl [--connect-timeout SECONDS] [--max-time SECONDS] [--retry N] [-x PROXY] [--limit-rate BYTES] [-k] URL
curl --help [all]

Options

-s
silent: no progress meter, and no error messages either
--help
list the common options; --help all lists every one
-i
print the response's status line and headers before the page
-I
ask for the headers only (a HEAD request)
-v
verbose: each step, the request (>) and the response (<) headers
-L
follow redirects to their Location
-f
fail on an error status (4xx, 5xx): exit code 22 and no page
-o page.html
write the body to the file after it instead of the screen; -o /dev/null throws it away
-w "%{http_code}\n"
once the transfer is done, print the format after it with its %{…} filled in: status, times, sizes, speed
-O
save the body in a file named after the URL's last part
-H "Accept: application/json"
add the header after it to the request (Accept, Content-Type, Host, Authorization…)
-X POST
send the request with the method after it (POST, PUT, DELETE…) instead of GET
-d DATA
send the text after it as the request's body, which makes the request a POST
-u user:password
log in with the user name and password after it (HTTP Basic)
--resolve blog.lab:80:10.0.2.10
connect to the address given for that name and port (NAME:PORT:ADDRESS), bypassing DNS
-c jar.txt
save the cookies the server sets into the file after it, a cookie jar
-b jar.txt
send the cookies kept in the file after it (or a name=value string)
-A "Mozilla/5.0"
send the text after it as the User-Agent
-e http://blog.lab/
send the URL after it as the Referer: the page you came from
-S
with -s, still print the error, with its exit code
--max-time 10 / -m 10
the most seconds the whole transfer may take before curl gives up, exit code 28
--connect-timeout 2
the most seconds connecting may take, and only connecting
--retry 2
how many more times to try after a failure that may pass (a timeout, a 5xx)
-x http://proxy:3128
send the request through the web proxy after it (CONNECT for https)
--limit-rate 100k
the most bytes a second to download (k and M for kilobytes and megabytes)
-k
accept any certificate (only to look, never to trust)
--socks5-hostname proxy.lab:1080
connect through the SOCKS 5 proxy after it, which also looks up the name
-4 / -6
IPv4 or IPv6 only
--http1.1 / --http2
offer only HTTP/1.1, or HTTP/2 first, in TLS's ALPN
--http3 / --http3-only
try HTTP/3 over QUIC with TCP raced beside it / QUIC or fail
-Z
run several transfers in parallel
-T FILE
upload a file (HTTP PUT, FTP STOR)
-P - / --disable-epsv
FTP: active mode (PORT) / classic passive PASV rather than EPSV

curl examples

curl -s http://webServer.lab/
Fetches the page and prints it.
curl -sv http://webServer.lab/
Fetches the page and shows every step: connecting, the request and the response headers.
curl -sI http://webServer.lab/
Fetches only the response headers.
curl -s -m 5 -o /dev/null -w '%{http_code} %{time_total}\n' http://webServer.lab/
Prints only the status code and total time, giving up after five seconds.
curl -s -m 5 http://10.0.2.10:81/; echo exit=$?
Tries a port and prints curl's exit code, which names what failed.
curl -sv https://bank.lab/ 2>&1 | grep -iE 'subject|issuer|expire'
Shows who the server's certificate names and who signed it.
curl -s -x http://webProxy.lab:3128 http://newsSite.lab/
Fetches the page through a forward proxy, which makes the request on your behalf.
curl -sL http://example.com/
Follows redirects (301, 302, 307, 308) to the final page instead of printing the redirect itself.
curl -o page.html https://example.com/
Saves the body to page.html instead of printing it.
curl -O https://example.com/files/netlab.iso
Saves the file under its own name from the URL, netlab.iso.
curl -C - -O https://example.com/files/netlab.iso
Resumes a download that broke off, from where the partial file ends.
curl --json '{"name":"edgeRouter"}' https://api.example.com/devices
POSTs JSON with the Content-Type and Accept headers set for you (curl 7.82 and later).
curl -H 'Content-Type: application/json' -d '{"name":"edgeRouter"}' https://api.example.com/devices
The same POST on any curl: -d makes the request a POST and sends the text as the body.
curl -d 'user=ana&pass=secret' https://example.com/login
Sends a form the way a browser does, as application/x-www-form-urlencoded.
curl -F 'file=@frr.conf' https://example.com/upload
Uploads a file as a multipart form, like a file input on a web page.
curl -H "Authorization: Bearer $TOKEN" https://api.example.com/me
Adds a header to the request: here the token most APIs ask for.
curl -u admin:netlab http://10.0.1.10:8080/api/devices
Logs in with HTTP basic authentication; leave out :netlab and curl asks for the password.
curl --resolve shop.lab:443:10.0.2.20 https://shop.lab/
Connects to 10.0.2.20 for shop.lab without touching DNS, keeping the right name for TLS and the Host header.
curl -s -o /dev/null -w 'dns %{time_namelookup} connect %{time_connect} tls %{time_appconnect} first byte %{time_starttransfer} total %{time_total}\n' https://example.com/
Times each phase of the request, every figure in seconds from the start: a slow DNS, connect or server shows on its own.
dns 0.004 connect 0.021 tls 0.058 first byte 0.142 total 0.143
curl -fsS https://example.com/health || echo down
Fails with exit code 22 on an HTTP error status (4xx, 5xx) instead of printing the error page: right for scripts.

Reading the output

$ curl -sv http://10.0.2.10/
*   Trying 10.0.2.10:80...
* Connected to 10.0.2.10 (10.0.2.10) port 80
> GET / HTTP/1.1
> Host: 10.0.2.10
> User-Agent: curl/8.14.1
>
< HTTP/1.1 200 OK
< Content-type: text/html
< Content-Length: 21
<
hello from webServer
  • * Trying then * Connected: the TCP handshake worked. Stuck at Trying: packets are dropped. Connection refused: nothing listens.
  • Lines with >: the request curl sent. Host: is the name the server sees.
  • Lines with <: the response. 200 is success; 3xx a redirect, 4xx your request, 5xx the server (a 503 from a balancer: no healthy server behind it).

Pitfalls

  • Without -m, curl may wait minutes on a dropped connection.
  • -k hides exactly the warning a man-in-the-middle triggers: use it to look, not to trust.

Run it on real machines

Learn it on real machines.

Every lab is a small network of real Linux machines in your browser. Start with a free account: Google or email, no card.

Start free