Cisco IOS-XE Device Health Check#

Structured triage procedure for assessing Cisco IOS-XE device health. Produces a prioritized findings report with severity classifications and recommended actions.

When to Use#

• Device is reported as slow, unresponsive, or dropping traffic
• Scheduled health audit of IOS-XE routers or switches
• Post-change verification after configuration or software updates
• Capacity planning data collection for CPU, memory, and interface utilization
• Incident response when a device is suspected as the fault domain

Prerequisites#

• SSH or console access to the target IOS-XE device (privilege level 1 minimum)
• Device running IOS-XE 16.x or 17.x (commands validated against 17.3+)
• Network reachability confirmed (ping/traceroute to management IP succeeds)
• Knowledge of the device's normal baseline (typical CPU, memory, traffic levels)
• Change control approval if performing checks during a maintenance window

Procedure#

Follow this sequence. Each step produces data for the final report. Do not skip steps unless the device is unresponsive (jump to Step 6 for crash recovery).

Step 1: Establish Baseline Context#

Collect device identity and uptime to frame the health check.

text

show version | include uptime|Version|bytes of memory
show inventory | include PID
show clock

Record: hostname, software version, uptime, hardware model, current time. Flag if uptime is unexpectedly short — indicates recent reload or crash.

Step 2: CPU Utilization Assessment#

text

show processes cpu sorted | head 20
show processes cpu history
show processes cpu platform sorted 5sec

Compare 5-second, 1-minute, and 5-minute averages against thresholds. If 5-second average exceeds 80%, identify the top process immediately.

Key processes to watch:

• IP Input — high values indicate traffic processing overload
• Crypto IKMP — VPN negotiation storms
• SNMP ENGINE — aggressive polling
• BGP Router — large table churn or route oscillation
• IOSD — general control plane congestion

Step 3: Memory Utilization Assessment#

text

show memory statistics
show memory platform information
show processes memory sorted | head 15

Calculate used percentage: (Total - Free) / Total * 100. Check for memory fragmentation: compare Largest Free block to Total Free. If largest free block is less than 10% of total free, fragmentation is a concern.

Step 4: Interface Health#

text

show interfaces summary
show interfaces counters errors
show interfaces | include line protocol|drops|error|CRC|collision

For each interface with errors:

• Calculate error rate: errors / (input packets + output packets) * 100
• Error rate above 0.1% is warning, above 1% is critical
• CRC errors suggest Layer 1 issues (cabling, optics, SFP)
• Input errors with no CRC suggest buffer or overrun issues
• Output drops indicate congestion — check QoS policy

Step 5: Routing Table Health#

text

show ip route summary
show ip bgp summary (if BGP is configured)
show ip ospf neighbor (if OSPF is configured)
show ip eigrp neighbors (if EIGRP is configured)

Verify: expected number of routes present, no unexpected route withdrawals, all routing protocol neighbors in established/full state.

Flag: neighbor state changes in the last hour, route count significantly different from baseline, any routes via unexpected next-hops.

Step 6: Platform and Environment#

text

show environment all
show platform software status control-processor brief
show logging | include %|Error|Warning|traceback (last 50 lines)

Check: power supply status, fan status, temperature readings. Any environmental alarm is an immediate escalation trigger. Review recent syslog for crash signatures (traceback, CPUHOG, MALLOCFAIL).

Threshold Tables#

Reference: references/threshold-tables.md for detailed per-parameter thresholds.

ParameterNormalWarningCriticalCPU 5-min avg< 40%40–70%> 70%CPU 5-sec spike< 80%80–90%> 90%Memory used< 70%70–85%> 85%Memory fragmentation> 10% largest/total5–10%< 5%Interface error rate< 0.01%0.01–0.1%> 0.1%Interface output drops< 100/hr100–1000/hr> 1000/hrRouting neighborsAll establishedFlappingDownTemperatureWithin specWithin 5°C of maxAt or above max

Decision Trees#

Triage Priority#

text

Is the device reachable?
├── No → Escalate immediately. Check console access, power, environment.
└── Yes
    ├── CPU critical? → Identify top process → Apply mitigation per process
    │   ├── IP Input → Check for traffic storm, ACL optimization
    │   ├── BGP Router → Check for route churn, peer flap, table size
    │   └── Other → Collect 'show tech-support' for TAC escalation
    ├── Memory critical? → Check for memory leak
    │   ├── Largest free < 5% of total → Likely fragmentation, schedule reload
    │   └── Steady growth over time → Memory leak, collect 'show mem alloc'
    ├── Interface errors? → Classify error type
    │   ├── CRC/input errors → Layer 1 (cable, optic, SFP)
    │   └── Output drops → QoS policy or congestion
    └── All within thresholds → Document clean health, schedule next check

Escalation Criteria#

Escalate to senior engineer or TAC when any of these conditions are met:

• CPU sustained above 90% for more than 15 minutes with no identifiable cause
• Memory below 15% free with no recent change to explain consumption
• Traceback or CPUHOG messages in logs within last 24 hours
• Environmental alarm (power, fan, temperature) present
• More than 3 routing neighbor state changes in last hour

Report Template#

Generate a structured report with these sections:

text

DEVICE HEALTH REPORT
====================
Device: [hostname]
Model: [PID from inventory]
Software: [version]
Uptime: [uptime string]
Check Time: [timestamp]
Performed By: [operator/agent]

SUMMARY: [HEALTHY | WARNING | CRITICAL]

FINDINGS:
1. [Severity] [Component] — [Description]
   Observed: [metric value]
   Threshold: [normal/warning/critical range]
   Action: [recommended action]

2. ...

RECOMMENDATIONS:
- [Prioritized list of actions]

NEXT CHECK: [scheduled date based on findings severity]

Severity levels for findings:

• INFO — within normal thresholds, noted for baseline
• WARNING — approaching threshold, monitor closely
• CRITICAL — threshold exceeded, action required
• EMERGENCY — device at risk of failure, immediate action

Troubleshooting#

Device Unresponsive to SSH#

Try console access. If console is also unresponsive, check power and environment remotely (smart PDU, out-of-band management). If the device has crashed, collect crashinfo: dir crashinfo: after recovery.

CPU Spikes During Health Check#

SNMP polling or show commands themselves can briefly spike CPU. Wait 30 seconds after connecting before collecting CPU data. Use terminal length 0 to avoid paging pauses that extend session time.

Inconsistent Memory Readings#

Memory values fluctuate during normal operation. Collect three samples at 30-second intervals and average them. Check show memory dead for memory that is allocated but unreachable (leak indicator).

Interface Counter Interpretation#

Counters are cumulative since last clear. Use show interfaces [name] to see the last clear time. For rate calculations, collect counters twice with a known interval: (counter2 - counter1) / interval_seconds.

Routing Protocol Neighbor Issues#

If OSPF neighbors are stuck in INIT/2WAY, check MTU mismatch and area configuration. If BGP peers show "Active" state, verify TCP connectivity on port 179 and check for ACL blocking. EIGRP stuck-in-active indicates a convergence problem downstream.