Day-to-day operational visibility: device-management protocols, syslog & conditional debug, SNMPv2c vs SNMPv3 (authPriv), Flexible NetFlow (Record / Exporter / Monitor), IP SLA (icmp-echo & udp-jitter) with track objects and EEM, IPv4 DHCP (server, relay, option 82), and IPv6 DHCP (stateless, stateful, prefix delegation).
4.1 Device-management diagnostics
Before SNMP and telemetry, every troubleshooting session starts with the basics: ping, traceroute, show, and debug.
Diagnostic command toolkit
| Command | What it tells you |
|---|---|
ping x.x.x.x source Loopback0 size 1500 df-bit |
L3 reachability + MTU/PMTUD validation |
traceroute x.x.x.x |
Per-hop latency & path; uses UDP 33434+ by default |
show ip cef x.x.x.x detail |
FIB programming for a destination |
show ip route x.x.x.x |
RIB selection, AD/metric, last-update timer |
show interfaces / summary |
Counters, errors, drops, queues |
debug ... |
Real-time troubleshooting (use sparingly!) |
debug condition interface ... / debug condition ip ... |
Conditional debug — restrict noisy debugs |
terminal monitor |
Send debug/console output to your vty session |
debug ip packet can crash a busy router. Set a condition first, then enable the debug, then unset the condition / undebug all when done.4.2 Syslog — severity, facility, conditional debug
The 8 syslog severity levels (memorize)
| Number | Keyword | Meaning |
|---|---|---|
| 0 | emergency | System unusable |
| 1 | alert | Action must be taken immediately |
| 2 | critical | Critical conditions |
| 3 | error | Error conditions |
| 4 | warning | Warnings |
| 5 | notification | Normal but significant (e.g. interface state changes) |
| 6 | informational | Informational messages |
| 7 | debugging | Debug-level (most verbose) |
Configuring a level X means everything from 0 to X is sent. So logging trap 6 sends 0-6 (everything except debug).
Mnemonic
Anatomy of a syslog message
timestamp ^facility-^severity^mnemonic description
Configuration template
R1(config)# service sequence-numbers
R1(config)# logging buffered 64000 informational
R1(config)# logging host 10.0.0.50 transport udp port 514
R1(config)# logging trap notifications ! 0-5 to remote server
R1(config)# logging console critical ! quiet console
R1(config)# logging origin-id hostname
! Conditional debug example: only debug OSPF for a specific neighbor
R1# debug condition ip 10.1.1.2
R1# debug ip ospf adj
! … troubleshoot …
R1# undebug all
4.3 SNMPv2c vs SNMPv3
Comparison
| SNMPv2c | SNMPv3 | |
|---|---|---|
| Auth model | Community string (cleartext) | User-based (USM): user/group/view |
| Encryption | None | DES / 3DES / AES-128/192/256 |
| Integrity | None | HMAC-MD5 / HMAC-SHA |
| Transport | UDP/161 (get), UDP/162 (trap) | UDP/161 / 162 |
| Best for | Lab / legacy | Production — required by ENARSI v1.1 |
SNMPv3 security levels
| Level | Authentication | Encryption (Privacy) |
|---|---|---|
| noAuthNoPriv | Username only | None |
| authNoPriv | HMAC-MD5 / HMAC-SHA | None |
| authPriv | HMAC-MD5 / HMAC-SHA | DES / 3DES / AES-128/192/256 |
SNMPv3 building blocks
Configuration template — authPriv
R1(config)# snmp-server view RO_VIEW iso included
R1(config)# snmp-server view RO_VIEW internet.6.3.15 excluded ! hide USM-MIB
! 2) Define a GROUP that uses authPriv security level + the view
R1(config)# snmp-server group MON_GRP v3 priv read RO_VIEW
! 3) Define a USER in that group, with HMAC-SHA + AES-256
R1(config)# snmp-server user nmsuser MON_GRP v3 \
auth sha SnmpAuth!Pwd priv aes 256 SnmpPriv!Pwd
! 4) Optional: send v3 traps to NMS
R1(config)# snmp-server host 10.0.0.50 version 3 priv nmsuser
R1# show snmp user
R1# show snmp group
R1# show snmp view
RO vs RW community on the wrong ACL).4.4 Flexible NetFlow — Record, Exporter, Monitor
NetFlow gives you a per-flow log of every conversation through an interface (5-tuple + counters). Flexible NetFlow (FNF) uses three configurable objects:
The three FNF objects
Configuration walkthrough
R1(config)# flow record FNF_REC
R1(config-flow-record)# match ipv4 source address
R1(config-flow-record)# match ipv4 destination address
R1(config-flow-record)# match ipv4 protocol
R1(config-flow-record)# match transport source-port
R1(config-flow-record)# match transport destination-port
R1(config-flow-record)# match interface input
R1(config-flow-record)# collect counter bytes
R1(config-flow-record)# collect counter packets
R1(config-flow-record)# collect timestamp sys-uptime first
R1(config-flow-record)# collect timestamp sys-uptime last
! Step 2 – flow exporter (where to send it)
R1(config)# flow exporter FNF_EXP
R1(config-flow-exporter)# destination 10.0.0.60
R1(config-flow-exporter)# source Loopback0
R1(config-flow-exporter)# transport udp 9996
R1(config-flow-exporter)# template data timeout 60
R1(config-flow-exporter)# export-protocol netflow-v9 ! or ipfix
! Step 3 – flow monitor (binds record + exporter)
R1(config)# flow monitor FNF_MON
R1(config-flow-monitor)# record FNF_REC
R1(config-flow-monitor)# exporter FNF_EXP
R1(config-flow-monitor)# cache timeout active 60
R1(config-flow-monitor)# cache timeout inactive 15
! Step 4 – apply to interface (ingress and/or egress)
R1(config)# interface GigabitEthernet0/1
R1(config-if)# ip flow monitor FNF_MON input
R1(config-if)# ip flow monitor FNF_MON output
R1# show flow monitor FNF_MON cache
R1# show flow exporter FNF_EXP statistics
4.5 IP SLA + Track Objects + EEM
IP SLA turns the router into an active probe. It generates synthetic traffic (ICMP, UDP, TCP, HTTP, DNS) and records latency, jitter, and reachability. Combined with track objects (consume the SLA result) and EEM (act on a state change), IP SLA becomes a closed-loop monitoring + reaction system.
Common IP SLA operation types
| Operation | What it measures | Needs Responder? |
|---|---|---|
| icmp-echo | Round-trip latency, reachability | No |
| udp-jitter | RTT, one-way delay, jitter, packet loss (VoIP-style) | Yes (timestamps both ends) |
| udp-echo | UDP RTT | Recommended |
| tcp-connect | TCP handshake time | No (target listens on port) |
| http | HTTP GET response time, DNS, TCP | No |
| dns | DNS query latency | No |
| path-jitter | Per-hop jitter via traceroute path | No (uses ICMP) |
The SLA + Track + EEM chain
Configuration template
R1(config)# ip sla 10
R1(config-ip-sla)# icmp-echo 8.8.8.8 source-interface Gi0/0
R1(config-ip-sla)# threshold 200
R1(config-ip-sla)# timeout 1000
R1(config-ip-sla)# frequency 5
R1(config)# ip sla schedule 10 life forever start-time now
! —– udp-jitter (needs responder on the far side) —–
R1(config)# ip sla 20
R1(config-ip-sla)# udp-jitter 10.2.2.2 16384 num-packets 100 interval 20
R1(config-ip-sla)# frequency 30
R1(config)# ip sla schedule 20 life forever start-time now
! —– Far-side responder —–
R2(config)# ip sla responder
! —– Track object linked to SLA —–
R1(config)# track 1 ip sla 10 reachability
R1(config-track)# delay down 10 up 5
! —– Use track on a static route (PBR-friendly failover) —–
R1(config)# ip route 0.0.0.0 0.0.0.0 198.51.100.1 track 1
R1(config)# ip route 0.0.0.0 0.0.0.0 203.0.113.1 200 ! floating backup
! —– EEM applet that reacts to SLA failure —–
R1(config)# event manager applet WAN_DOWN
R1(config-applet)# event track 1 state down
R1(config-applet)# action 1.0 syslog msg “Primary WAN failed – failing over”
R1(config-applet)# action 2.0 cli command “enable”
R1(config-applet)# action 3.0 cli command “show ip route 0.0.0.0”
R1# show ip sla statistics
R1# show track 1
R1# show event manager policy registered
4.6 DHCPv4 — server, relay, Option 82
The four-message DORA exchange
IOS DHCPv4 server configuration
R1(config)# ip dhcp excluded-address 10.10.10.250 10.10.10.254
R1(config)# ip dhcp pool USERS_VLAN10
R1(dhcp-config)# network 10.10.10.0 /24
R1(dhcp-config)# default-router 10.10.10.1
R1(dhcp-config)# dns-server 10.0.0.53 10.0.0.54
R1(dhcp-config)# domain-name lab.local
R1(dhcp-config)# lease 0 8 0 ! 8 hours
R1(dhcp-config)# option 150 ip 10.0.0.30 10.0.0.31 ! TFTP for IP phones
! Reservation by client-id
R1(config)# ip dhcp pool PRINTER
R1(dhcp-config)# host 10.10.10.50 /24
R1(dhcp-config)# client-identifier 0100.5056.aa.bb.cc
R1# show ip dhcp binding
R1# show ip dhcp pool
R1# show ip dhcp conflict
DHCP relay (ip helper-address)
When the DHCP server is in another subnet, the access-layer L3 device acts as a relay: it converts the client’s broadcast DISCOVER into a unicast forwarded message to the configured server, with giaddr set so the server picks the right pool.
SVI(config-if)# ip address 10.10.10.1 255.255.255.0
SVI(config-if)# ip helper-address 10.0.0.20
! Multiple servers? Just add more helpers (each gets a copy)
SVI(config-if)# ip helper-address 10.0.0.21
ip helper-address actually forward? Eight UDP services by default: TFTP(69), Domain(53), Time(37), TACACS(49), DNS(53), BootP server(67), BootP client(68), NetBIOS NS/DS(137/138). Use no ip forward-protocol udp ... to trim what is unintentionally forwarded.Option 82 (Relay Agent Information)
The relay agent inserts circuit-ID (which port/VLAN the request came from) and remote-ID (which switch) into the DHCP request. The server can then use this to assign a specific subnet, log the source port, or enforce policy.
SW(config)# ip dhcp snooping vlan 10,20
SW(config)# ip dhcp snooping information option ! insert option 82
SW(config)# interface Gi1/0/24
SW(config-if)# ip dhcp snooping trust ! uplink toward server
4.7 DHCPv6 — stateless, stateful, prefix delegation
IPv6 has two ways to assign addresses: SLAAC (stateless auto-config from a router advertisement) and DHCPv6. ENARSI tests three flavors of DHCPv6:
The three DHCPv6 flavors
| Mode | What client gets | RA flags | Use case |
|---|---|---|---|
| Stateless DHCPv6 | Other config (DNS, NTP, domain) only — address is from SLAAC | O=1, M=0, A=1 | Auto-addressed hosts that still need DNS info |
| Stateful DHCPv6 | Full IPv6 address + other options (server tracks bindings) | M=1, O=1, A=0 | Enterprise hosts (deterministic addresses) |
| Prefix Delegation (PD) | An entire prefix (e.g. /56 or /60) for the client router to subnet downstream | (Server-only feature) | SP-CPE hand-off; SOHO routers |
Stateless DHCPv6 server (most common)
R1(config)# ipv6 dhcp pool STATELESS_POOL
R1(config-dhcpv6)# dns-server 2001:db8:0:53::1
R1(config-dhcpv6)# domain-name lab.local
R1(config)# interface Gi0/1
R1(config-if)# ipv6 address 2001:db8:10::1/64
R1(config-if)# ipv6 nd other-config-flag ! O=1
R1(config-if)# ipv6 dhcp server STATELESS_POOL
Stateful DHCPv6 server
R1(config-dhcpv6)# address prefix 2001:db8:10::/64 lifetime 3600 1800
R1(config-dhcpv6)# dns-server 2001:db8:0:53::1
R1(config-dhcpv6)# domain-name lab.local
R1(config)# interface Gi0/1
R1(config-if)# ipv6 nd managed-config-flag ! M=1
R1(config-if)# ipv6 nd ra suppress ! optional: RA off
R1(config-if)# ipv6 dhcp server STATEFUL_POOL
Prefix delegation (server side)
R1(config)# ipv6 dhcp pool PD_POOL
R1(config-dhcpv6)# prefix-delegation pool ENT_PREFIXES lifetime 86400 43200
R1(config-dhcpv6)# dns-server 2001:db8:0:53::1
R1(config)# interface Gi0/1
R1(config-if)# ipv6 dhcp server PD_POOL
Prefix delegation (client / CPE side)
CPE(config-if)# description WAN
CPE(config-if)# ipv6 enable
CPE(config-if)# ipv6 dhcp client pd PREFIX_FROM_SP
CPE(config)# interface Gi0/1
CPE(config-if)# description LAN
CPE(config-if)# ipv6 address PREFIX_FROM_SP ::1:0:0:0:1/64 ! sub-allocate from delegated prefix
CPE(config-if)# ipv6 nd other-config-flag
CPE# show ipv6 dhcp interface
CPE# show ipv6 general-prefix
DHCPv6 relay (server in another subnet)
SVI(config-if)# ipv6 dhcp relay destination 2001:db8:0:dh::1
Hands-on labs (8)
R1(config)# service timestamps log datetime msec localtime show-timezone
R1(config)# service sequence-numbers
R1(config)# logging buffered 64000 informational
R1(config)# logging origin-id hostname
R1(config)# logging host 10.0.0.50 transport udp port 514
R1(config)# logging trap notifications
R1(config)# logging console critical
R1(config)# logging userinfo
! Conditional debug: only one peer
R1# debug condition ip 10.1.1.2
R1# debug ip ospf adj
R1# show debug condition
R1# undebug all
R1# show logging | include CONDITION
R1(config-std-nacl)# permit host 10.0.0.50
R1(config)# snmp-server view RO_VIEW iso included
R1(config)# snmp-server view RO_VIEW internet.6.3.15 excluded
R1(config)# snmp-server group MON_GRP v3 priv read RO_VIEW access SNMP_NMS
R1(config)# snmp-server user nmsuser MON_GRP v3 \
auth sha SnmpAuth!Pwd priv aes 256 SnmpPriv!Pwd
R1(config)# snmp-server contact [email protected]
R1(config)# snmp-server location DC-A Rack 12
R1(config)# snmp-server enable traps config bgp ospf isis snmp linkup linkdown
R1(config)# snmp-server host 10.0.0.50 version 3 priv nmsuser config bgp
R1# show snmp user
R1# show snmp group
R1# show snmp host
! From the NMS:
$ snmpwalk -v3 -l authPriv -u nmsuser -a SHA -A SnmpAuth!Pwd \
-x AES -X SnmpPriv!Pwd 10.1.1.1 sysName
R1(config-flow-record)# match ipv4 source address
R1(config-flow-record)# match ipv4 destination address
R1(config-flow-record)# match ipv4 protocol
R1(config-flow-record)# match ipv4 tos
R1(config-flow-record)# match transport source-port
R1(config-flow-record)# match transport destination-port
R1(config-flow-record)# match interface input
R1(config-flow-record)# collect interface output
R1(config-flow-record)# collect counter bytes long
R1(config-flow-record)# collect counter packets long
R1(config-flow-record)# collect timestamp sys-uptime first
R1(config-flow-record)# collect timestamp sys-uptime last
R1(config)# flow exporter FNF_EXP
R1(config-flow-exporter)# destination 10.0.0.60
R1(config-flow-exporter)# source Loopback0
R1(config-flow-exporter)# transport udp 9996
R1(config-flow-exporter)# template data timeout 60
R1(config-flow-exporter)# export-protocol ipfix
R1(config)# flow monitor FNF_MON
R1(config-flow-monitor)# record FNF_REC
R1(config-flow-monitor)# exporter FNF_EXP
R1(config-flow-monitor)# cache timeout active 60
R1(config)# interface GigabitEthernet0/0
R1(config-if)# ip flow monitor FNF_MON input
R1(config-if)# ip flow monitor FNF_MON output
R1# show flow monitor FNF_MON cache format table
R1# show flow exporter statistics
R1# show flow interface
R2(config)# ip sla responder
! —– R1 (source) —–
R1(config)# ip sla 100
R1(config-ip-sla)# udp-jitter 10.2.2.2 16384 num-packets 100 interval 20
R1(config-ip-sla)# request-data-size 160
R1(config-ip-sla)# tos 184
R1(config-ip-sla)# threshold 30
R1(config-ip-sla)# frequency 30
R1(config)# ip sla schedule 100 life forever start-time now
R1(config)# track 1 ip sla 100 reachability
R1(config-track)# delay down 10 up 5
R1(config)# ip route 0.0.0.0 0.0.0.0 198.51.100.1 track 1
R1(config)# ip route 0.0.0.0 0.0.0.0 203.0.113.1 200 ! floating backup
R1(config)# event manager applet WAN_FAILOVER
R1(config-applet)# event track 1 state down
R1(config-applet)# action 1.0 syslog priority warnings msg “Primary WAN SLA failed – failing over”
R1(config-applet)# action 2.0 cli command “enable”
R1(config-applet)# action 3.0 cli command “show ip route 0.0.0.0”
R1# show ip sla statistics 100 details
R1# show track 1
R1# show event manager policy registered
R1(config)# ip dhcp excluded-address 10.10.10.1 10.10.10.20
R1(config)# ip dhcp excluded-address 10.10.10.250 10.10.10.254
R1(config)# ip dhcp pool VLAN10
R1(dhcp-config)# network 10.10.10.0 /24
R1(dhcp-config)# default-router 10.10.10.1
R1(dhcp-config)# dns-server 10.0.0.53
R1(dhcp-config)# domain-name lab.local
R1(dhcp-config)# lease 0 8 0
R1(dhcp-config)# option 150 ip 10.0.0.30 10.0.0.31
! Reservation
R1(config)# ip dhcp pool PRINTER
R1(dhcp-config)# host 10.10.10.50 /24
R1(dhcp-config)# client-identifier 0100.5056.aabb.cc
! —– Access switch SVI acting as relay —–
SW(config)# interface Vlan10
SW(config-if)# ip address 10.10.10.1 255.255.255.0
SW(config-if)# ip helper-address 10.0.0.20
SW(config-if)# ip dhcp relay information trusted ! accept option 82
R1# show ip dhcp binding
R1# show ip dhcp pool
R1# show ip dhcp conflict
R1(config)# ipv6 dhcp pool DNS_POOL
R1(config-dhcpv6)# dns-server 2001:db8:0:53::1
R1(config-dhcpv6)# domain-name lab.local
R1(config)# interface Gi0/1
R1(config-if)# ipv6 address 2001:db8:10::1/64
R1(config-if)# ipv6 enable
R1(config-if)# ipv6 nd other-config-flag
R1(config-if)# ipv6 dhcp server DNS_POOL
R1# show ipv6 dhcp pool
R1# show ipv6 dhcp interface Gi0/1
! On a host you should see SLAAC address + DNS via DHCPv6
R1(config-dhcpv6)# address prefix 2001:db8:10::/64 lifetime 3600 1800
R1(config-dhcpv6)# dns-server 2001:db8:0:53::1
R1(config)# interface Gi0/1
R1(config-if)# ipv6 nd managed-config-flag
R1(config-if)# ipv6 nd prefix 2001:db8:10::/64 no-autoconfig
R1(config-if)# ipv6 dhcp server STATEFUL_POOL
R1# show ipv6 dhcp binding
R1# show ipv6 dhcp pool
SP(config)# ipv6 local pool ENT_PREFIXES 2001:db8:1100::/40 56
SP(config)# ipv6 dhcp pool PD_POOL
SP(config-dhcpv6)# prefix-delegation pool ENT_PREFIXES lifetime 86400 43200
SP(config-dhcpv6)# dns-server 2001:db8:0:53::1
SP(config)# interface Gi0/0
SP(config-if)# ipv6 dhcp server PD_POOL
! —– CPE (client) —–
CPE(config)# interface Gi0/0
CPE(config-if)# description WAN
CPE(config-if)# ipv6 enable
CPE(config-if)# ipv6 dhcp client pd PREFIX_FROM_SP
CPE(config)# interface Gi0/1
CPE(config-if)# description LAN-A
CPE(config-if)# ipv6 address PREFIX_FROM_SP ::1:0:0:0:1/64
CPE(config)# interface Gi0/2
CPE(config-if)# description LAN-B
CPE(config-if)# ipv6 address PREFIX_FROM_SP ::2:0:0:0:1/64
CPE# show ipv6 dhcp interface
CPE# show ipv6 general-prefix
CPE# show ipv6 interface brief
Check Your Understanding
Twenty-five questions on this section. Each answer is explained as you go.
