In-band and out-of-band management

In-band management (SSH to a router's ordinary address) is free and simple, and lost whenever the network is: a wrong shutdown or ACL cuts the path you would fix it over. Out-of-band management gives each device a management port on a management network of its own, reached through a jump box, and keeps it out of the data plane: the port does not route (a management VRF on real gear) and the management plane listens only there. It survives data-network failures, not a device that refuses everyone: for that there is the console port.

How out-of-band management works, step by step

In the out-of-band management lab coreRouter routes officeNetwork (10.0.1.0/24) to serverNetwork (10.0.2.0/24), and has a third card, its management port, at 10.0.3.2 on managementNetwork. adminLaptop has a card in the office (10.0.1.10) and one on the NOC network (10.0.4.10); jumpBox (10.0.4.2 and 10.0.3.10) is the only door into the management network.

1. In band. The admin SSHes to 10.0.1.2, the router's office address. The session shares the office link with user traffic.
2. One wrong command. Shutting the office card takes down the link the session rides on. The office loses the app server (the data plane) and the admin loses the router (the management plane, in band).
3. Out of band. The admin SSHes to jumpBox on the NOC network, then on to 10.0.3.2. That path never touches the office card, so it still works; the admin brings the card back up.
4. The management port is not data plane. coreRouter drops anything forwarded to or from 10.0.3.0/24, and answers 10.0.3.2 only from the management network. On real gear this is a management VRF with its own routing table (VRF); on Linux the same idea is a vrf device or firewall rules.
5. Management only there. sshd gets ListenAddress 10.0.3.2, so the office addresses refuse SSH.
6. The last way in. A firewall rule that drops SSH on every card locks out the management network too. Only the console port, a serial line with no address or firewall, still reaches the system.

So in-band fails with the network, out-of-band survives the data network, and the console survives the device's own rules.

What it looks like on a Linux machine

Through the jump box, the office card is seen down (ip -br prints name, state, MAC, flags):

$ ssh admin@10.0.4.2 "ssh root@10.0.3.2 'ip -br link show dev officeNetwork'"
officeNetwork@if42 DOWN           6e:21:9a:4c:0b:12 <BROADCAST,MULTICAST>
$ ssh admin@10.0.4.2 "ssh root@10.0.3.2 'ip link set dev officeNetwork up'"

After ListenAddress 10.0.3.2 and a SIGHUP to sshd, it listens on the management address only:

$ ss -ltn sport = :22
State  Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0      128         10.0.3.2:22        0.0.0.0:*

From the office the router now refuses SSH at once, while the jump box still gets in:

$ ssh root@10.0.1.2 hostname
ssh: connect to host 10.0.1.2 port 22: Connection refused

On the console, the rule that locked everyone out is the first one:

$ iptables -S NETLAB-INPUT
-N NETLAB-INPUT
-A NETLAB-INPUT -p tcp -m tcp --dport 22 -j DROP
-A NETLAB-INPUT -s 10.0.3.0/24 -d 10.0.3.2/32 -j ACCEPT
-A NETLAB-INPUT -d 10.0.3.2/32 -j DROP
$ iptables -D NETLAB-INPUT -p tcp --dport 22 -j DROP

What goes wrong, and how you see it

Locked out in band. The session freezes after the command, and new logins time out: ssh: connect to host 10.0.1.2 port 22: Connection timed out. Go out of band. Before risky changes, schedule a rollback (a timed job that restores the config unless cancelled).

The management network routes into the data network. From the office, ping 10.0.3.2 answers. The management port is reachable by everyone it was meant to be hidden from. Drop forwarding to and from it, and answer its address only from the management side.

Management still listens everywhere. ss -ltn shows 0.0.0.0:22. Bind sshd, SNMP and web consoles to the management address.

A rule that blocks the management port too. Through the jump box, logins time out although the jump box itself works. The device refuses everyone; use the console, then put the management network's accept above any drop.

One door, no spare. If the jump box is the only way in, losing it loses everything. Keep a console server reachable by a separate path, such as a cellular modem.

Questions people ask

In-band vs out-of-band management? In band manages a device over the networks it carries; free, but lost with them. Out of band uses a separate management network or console line that survives data-network failures.

Is a jump box the same as a bastion host? Yes. The jump box here is a bastion host for the management network, and the bastion lab covers it in depth.

Do I still need a console port with OOB? Yes. The management network survives a broken data plane, not a device whose own firewall or config refuses every card. The console reaches it without crossing any card.

See it on real machines