high cpu utilization cisco
I need help on troubleshooting 'high cpu utilization' issue.
I've follow Cisco guidelines but I'm still having issues.
Anyone out there could assist please?
Cisco 2811
Version 12.4(24)T2
I've noticed one of the process is high 'IP Input'.
Router1#sh processes cpu sorte | ex 0.00
CPU utilization for five seconds: 70%/42%; one minute: 64%; five minutes: 59%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
104 60120280 44871638 1339 15.37% 14.26% 11.85% 0 IP Input
276 19482936 885164 22010 2.64% 1.85% 5.77% 0 SNMP ENGINE
163 1711248 103417269 16 2.08% 2.08% 1.84% 0 HQF Shaper Backg
274 1629536 1682225 968 1.76% 1.09% 0.49% 0 IP SNMP
261 2677980 14061400 190 1.52% 1.54% 1.52% 0 GLBP
19 3792636 6077592 624 0.63% 0.75% 0.73% 0 ARP Input
275 479980 846857 566 0.55% 0.33% 0.15% 0 PDU DISPATCHER
263 591024 27651611 21 0.47% 0.46% 0.44% 0 HSRP Common
262 2992532 9810618 305 0.47% 0.41% 0.40% 0 GLBP Input
257 1868628 497783 3753 0.39% 0.26% 0.28% 0 DHCPD Receive
254 532504 14192412 37 0.31% 0.31% 0.31% 0 PPP manager
131 1148080 726384 1580 0.31% 0.28% 0.29% 0 CEF: IPv4 proces
259 490716 9958006 49 0.31% 0.26% 0.24% 0 PIM Process
255 348636 14192429 24 0.23% 0.17% 0.16% 0 PPP Events
41 828300 474843 1744 0.23% 0.21% 0.22% 0 Per-Second Jobs
258 2089144 3347419 624 0.15% 0.29% 0.30% 0 IGMP Input
2 99912 94951 1052 0.07% 0.06% 0.06% 0 Load Meter
137 146956 2137787 68 0.07% 0.08% 0.07% 0 TCP Timer
164 107112 4565238 23 0.07% 0.07% 0.07% 0 RBSCP Background
111 3544 1381 2566 0.07% 0.04% 0.01% 514 SSH Process
260 129092 4631125 27 0.07% 0.07% 0.07% 0 Mwheel Process
Router1#
Router1sh processes cpu his
55555556666655555666665555
88888881111122222222223333
100
90
80
70
60 ************ ***** ***** *****
50 **************************
40 **************************
30 **************************
20 **************************
10 **************************
0....5....1....1....2....2
0 5 0 5 0 5 0 5 0 5 0
CPU% per second (last 60 seconds)
66687676699554655445788667
23133204066609185316391222
100 ** ***
90 #* * *#*
80 * #* ** *#* **
70 #* * ## *## **** * *#* ** *
60 **#####**##* *** *##########*#* * *##** ## * #
50 ###########***#** ##############*** *###** * ** ## *****##
40 #############*############
30 ##########################
20 ##########################
10 ##########################
0....5....1....1....2....2
0 5 0 5 0 5 0 5 0 5 0
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
99999999999999999999999999
96887889343276579679858898
100 ******** ******************* ********************** ********
90 **************************
80 **************************
70 **************************
60 #**#************#**##***##
50 ######*******#*######*####
40 ########***###############
30 ########*#################
20 ##########################
10 ##########################
0....5....1....1....2....2
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
Router1
I've follow Cisco guidelines but I'm still having issues.
Anyone out there could assist please?
Cisco 2811
Version 12.4(24)T2
I've noticed one of the process is high 'IP Input'.
Router1#sh processes cpu sorte | ex 0.00
CPU utilization for five seconds: 70%/42%; one minute: 64%; five minutes: 59%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
104 60120280 44871638 1339 15.37% 14.26% 11.85% 0 IP Input
276 19482936 885164 22010 2.64% 1.85% 5.77% 0 SNMP ENGINE
163 1711248 103417269 16 2.08% 2.08% 1.84% 0 HQF Shaper Backg
274 1629536 1682225 968 1.76% 1.09% 0.49% 0 IP SNMP
261 2677980 14061400 190 1.52% 1.54% 1.52% 0 GLBP
19 3792636 6077592 624 0.63% 0.75% 0.73% 0 ARP Input
275 479980 846857 566 0.55% 0.33% 0.15% 0 PDU DISPATCHER
263 591024 27651611 21 0.47% 0.46% 0.44% 0 HSRP Common
262 2992532 9810618 305 0.47% 0.41% 0.40% 0 GLBP Input
257 1868628 497783 3753 0.39% 0.26% 0.28% 0 DHCPD Receive
254 532504 14192412 37 0.31% 0.31% 0.31% 0 PPP manager
131 1148080 726384 1580 0.31% 0.28% 0.29% 0 CEF: IPv4 proces
259 490716 9958006 49 0.31% 0.26% 0.24% 0 PIM Process
255 348636 14192429 24 0.23% 0.17% 0.16% 0 PPP Events
41 828300 474843 1744 0.23% 0.21% 0.22% 0 Per-Second Jobs
258 2089144 3347419 624 0.15% 0.29% 0.30% 0 IGMP Input
2 99912 94951 1052 0.07% 0.06% 0.06% 0 Load Meter
137 146956 2137787 68 0.07% 0.08% 0.07% 0 TCP Timer
164 107112 4565238 23 0.07% 0.07% 0.07% 0 RBSCP Background
111 3544 1381 2566 0.07% 0.04% 0.01% 514 SSH Process
260 129092 4631125 27 0.07% 0.07% 0.07% 0 Mwheel Process
Router1#
Router1sh processes cpu his
55555556666655555666665555
88888881111122222222223333
100
90
80
70
60 ************ ***** ***** *****
50 **************************
40 **************************
30 **************************
20 **************************
10 **************************
0....5....1....1....2....2
0 5 0 5 0 5 0 5 0 5 0
CPU% per second (last 60 seconds)
66687676699554655445788667
23133204066609185316391222
100 ** ***
90 #* * *#*
80 * #* ** *#* **
70 #* * ## *## **** * *#* ** *
60 **#####**##* *** *##########*#* * *##** ## * #
50 ###########***#** ##############*** *###** * ** ## *****##
40 #############*############
30 ##########################
20 ##########################
10 ##########################
0....5....1....1....2....2
0 5 0 5 0 5 0 5 0 5 0
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
99999999999999999999999999
96887889343276579679858898
100 ******** ******************* ********************** ********
90 **************************
80 **************************
70 **************************
60 #**#************#**##***##
50 ######*******#*######*####
40 ########***###############
30 ########*#################
20 ##########################
10 ##########################
0....5....1....1....2....2
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
Router1
ASKER
Yes I made sure IP CEF was enable globally and per interface.
Router1#sh ip int multilink 1
Multilink1 is up, line protocol is up
Internet address is 172.16.20.218/30
Broadcast address is 255.255.255.255
Address determined by non-volatile memory
Peer address is 172.16.20.217
MTU is 1500 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Outgoing access list is not set
Inbound access list is not set
Proxy ARP is enabled
Local Proxy ARP is disabled
Security level is default
Split horizon is enabled
ICMP redirects are always sent
ICMP unreachables are always sent
ICMP mask replies are never sent
IP fast switching is enabled
IP fast switching on the same interface is enabled
IP Flow switching is disabled
IP CEF switching is enabled
IP CEF switching turbo vector
here the output of sh processes cpu sorte | ex _0.00%_.*_0.00%_.*_0.00%_
Router1#sh processes cpu sorte | ex _0.00%_.*_0.00%_.*_0.00%_
CPU utilization for five seconds: 67%/34%; one minute: 68%; five minutes: 65%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
104 60567472 45212161 1339 24.71% 23.99% 20.05% 0 IP Input
163 1722864 103843405 16 2.00% 1.91% 1.97% 0 HQF Shaper Backg
261 2691256 14122028 190 1.60% 1.52% 1.51% 0 GLBP
19 3813156 6109137 624 0.64% 0.62% 0.61% 0 ARP Input
262 3007680 9852905 305 0.40% 0.41% 0.40% 0 GLBP Input
263 593992 27770237 21 0.40% 0.41% 0.41% 0 HSRP Common
131 1155436 729529 1583 0.32% 0.28% 0.28% 0 CEF: IPv4 proces
254 535764 14253891 37 0.32% 0.30% 0.31% 0 PPP manager
259 493624 10001531 49 0.24% 0.25% 0.24% 0 PIM Process
9 793076 245180 3234 0.24% 0.12% 0.10% 0 BGP I/O
258 2099040 3362409 624 0.24% 0.25% 0.29% 0 IGMP Input
255 350572 14253908 24 0.16% 0.16% 0.16% 0 PPP Events
41 832752 476897 1746 0.16% 0.18% 0.20% 0 Per-Second Jobs
2 100524 95362 1054 0.08% 0.07% 0.08% 0 Load Meter
274 1637144 1689369 969 0.08% 0.09% 0.18% 0 IP SNMP
137 147856 2147795 68 0.08% 0.06% 0.07% 0 TCP Timer
256 12768 474460 26 0.08% 0.00% 0.00% 0 Multilink PPP
270 15264 47591 320 0.08% 0.00% 0.00% 0 MFI LFD Stats Pr
164 107736 4584947 23 0.08% 0.05% 0.07% 0 RBSCP Background
58 41696 1896800 21 0.08% 0.01% 0.00% 0 Netclock Backgro
285 248652 160933 1545 0.08% 0.03% 0.01% 0 BGP Router
276 19569276 888949 22014 0.08% 0.65% 2.74% 0 SNMP ENGINE
260 129884 4651203 27 0.08% 0.07% 0.07% 0 Mwheel Process
67 199804 31740 6295 0.00% 0.02% 0.00% 0 BGP Scanner
111 7708 2201 3502 0.00% 0.33% 0.27% 514 SSH Process
5 947236 65506 14460 0.00% 0.10% 0.11% 0 Check heaps
79 80184 1896810 42 0.00% 0.04% 0.05% 0 e1t1 Framer back
179 9128 4690 1946 0.00% 0.02% 0.00% 0 TPLUS
208 51332 2356981 21 0.00% 0.02% 0.00% 0 Atheros LED Ctro
249 262680 8064 32574 0.00% 0.02% 0.00% 0 Per-minute Jobs
257 1879360 500533 3754 0.00% 0.18% 0.24% 0 DHCPD Receive
264 128948 349035 369 0.00% 0.01% 0.00% 0 HSRP IPv4
275 482336 850439 567 0.00% 0.02% 0.04% 0 PDU DISPATCHER
280 33932 477341 71 0.00% 0.01% 0.00% 0 NTP
Router1#sh ip int multilink 1
Multilink1 is up, line protocol is up
Internet address is 172.16.20.218/30
Broadcast address is 255.255.255.255
Address determined by non-volatile memory
Peer address is 172.16.20.217
MTU is 1500 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Outgoing access list is not set
Inbound access list is not set
Proxy ARP is enabled
Local Proxy ARP is disabled
Security level is default
Split horizon is enabled
ICMP redirects are always sent
ICMP unreachables are always sent
ICMP mask replies are never sent
IP fast switching is enabled
IP fast switching on the same interface is enabled
IP Flow switching is disabled
IP CEF switching is enabled
IP CEF switching turbo vector
here the output of sh processes cpu sorte | ex _0.00%_.*_0.00%_.*_0.00%_
Router1#sh processes cpu sorte | ex _0.00%_.*_0.00%_.*_0.00%_
CPU utilization for five seconds: 67%/34%; one minute: 68%; five minutes: 65%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
104 60567472 45212161 1339 24.71% 23.99% 20.05% 0 IP Input
163 1722864 103843405 16 2.00% 1.91% 1.97% 0 HQF Shaper Backg
261 2691256 14122028 190 1.60% 1.52% 1.51% 0 GLBP
19 3813156 6109137 624 0.64% 0.62% 0.61% 0 ARP Input
262 3007680 9852905 305 0.40% 0.41% 0.40% 0 GLBP Input
263 593992 27770237 21 0.40% 0.41% 0.41% 0 HSRP Common
131 1155436 729529 1583 0.32% 0.28% 0.28% 0 CEF: IPv4 proces
254 535764 14253891 37 0.32% 0.30% 0.31% 0 PPP manager
259 493624 10001531 49 0.24% 0.25% 0.24% 0 PIM Process
9 793076 245180 3234 0.24% 0.12% 0.10% 0 BGP I/O
258 2099040 3362409 624 0.24% 0.25% 0.29% 0 IGMP Input
255 350572 14253908 24 0.16% 0.16% 0.16% 0 PPP Events
41 832752 476897 1746 0.16% 0.18% 0.20% 0 Per-Second Jobs
2 100524 95362 1054 0.08% 0.07% 0.08% 0 Load Meter
274 1637144 1689369 969 0.08% 0.09% 0.18% 0 IP SNMP
137 147856 2147795 68 0.08% 0.06% 0.07% 0 TCP Timer
256 12768 474460 26 0.08% 0.00% 0.00% 0 Multilink PPP
270 15264 47591 320 0.08% 0.00% 0.00% 0 MFI LFD Stats Pr
164 107736 4584947 23 0.08% 0.05% 0.07% 0 RBSCP Background
58 41696 1896800 21 0.08% 0.01% 0.00% 0 Netclock Backgro
285 248652 160933 1545 0.08% 0.03% 0.01% 0 BGP Router
276 19569276 888949 22014 0.08% 0.65% 2.74% 0 SNMP ENGINE
260 129884 4651203 27 0.08% 0.07% 0.07% 0 Mwheel Process
67 199804 31740 6295 0.00% 0.02% 0.00% 0 BGP Scanner
111 7708 2201 3502 0.00% 0.33% 0.27% 514 SSH Process
5 947236 65506 14460 0.00% 0.10% 0.11% 0 Check heaps
79 80184 1896810 42 0.00% 0.04% 0.05% 0 e1t1 Framer back
179 9128 4690 1946 0.00% 0.02% 0.00% 0 TPLUS
208 51332 2356981 21 0.00% 0.02% 0.00% 0 Atheros LED Ctro
249 262680 8064 32574 0.00% 0.02% 0.00% 0 Per-minute Jobs
257 1879360 500533 3754 0.00% 0.18% 0.24% 0 DHCPD Receive
264 128948 349035 369 0.00% 0.01% 0.00% 0 HSRP IPv4
275 482336 850439 567 0.00% 0.02% 0.04% 0 PDU DISPATCHER
280 33932 477341 71 0.00% 0.01% 0.00% 0 NTP
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Yes. Def 'Interrupts' are very high.
I'm looking to see what traffic is killing the cpu. Any idea?
I'm looking to see what traffic is killing the cpu. Any idea?
sh processes cpu sorte | ex 0.00
but
sh processes cpu sorte | ex _0.00%_.*_0.00%_.*_0.00%_
if any value in row is 0.00 - raw will not be shown.
Since IP Input CPU usage is high what is output from
sh ip interface <type X/Y>
Are interfaces using ip cef switching?