asked on

High CPU Utilization on Cisco 2960C Stack

Hi,

I am seeing this on both of my 2960 stacks:

CPU utilization for five seconds: 40%/13%; one minute: 56%; five minutes: 46%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
133 912176 88773 10275 16.29% 16.30% 16.41% 0 Hulc LED Process
COLMDPS01#sh processes cpu | exclude 0.00%
CPU utilization for five seconds: 32%/8%; one minute: 40%; five minutes: 44%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
11 7076 15291 462 0.19% 0.14% 0.13% 0 ARP Input
43 2175 948 2294 0.09% 0.03% 0.03% 0 Compute load avg
63 3248 33354 97 0.19% 0.06% 0.05% 0 RedEarth Tx Mana
103 46470 10011 4641 0.59% 0.81% 0.80% 0 hpm counter proc
104 3462 19965 173 0.19% 0.08% 0.08% 0 HRPC pm-counters
133 927885 90266 10279 16.65% 16.33% 16.39% 0 Hulc LED Process

IOS Version:

Switch Ports Model SW Version SW Image
------ ----- ----- ---------- ----------
* 1 52 WS-C2960S-48FPS-L 12.2(53)SE2 C2960S-UNIVERSALK9-M
2 52 WS-C2960S-48FPS-L 12.2(53)SE2 C2960S-UNIVERSALK9-M
3 52 WS-C2960S-48FPS-L 12.2(53)SE2 C2960S-UNIVERSALK9-M

I have done some troubleshooting at Cisco recommendation and the only thing I can find that may be a concern is this:

sh controllers cpu-interface
<snip>
cpu-queue-frames retrieved dropped invalid hol-block stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc 171278 0 0 0 0
stp 35627 0 4756 0 0

Initally I can't see any problem with RSTP across the network.

Does anyone have any suggestions?

Thanks.
D

Soulja

I would double check your connections, making sure there are not loops. Make sure bpdu guard is enabled on all access ports. Make sure you have your desired root for each of your vlans.

rochey2009

How many vlans do you have?

ncomper

ASKER

There are redundant links in the switched network.

STP has blocked the correct ports where there is redundancy.

There are no ports in the 'err-disabled' state.

The root bridge is set correctly for each vlan. (We are running PVST)

BDUGuard is running on all access ports. (Combined with portfast)

Last 60 minutes:

100
90 *
80 ********** ******************************** ***#*********
70 *########################################################
60 **########################################################
50 **########################################################
40 ##########################################################
30 ##########################################################
20 ##########################################################
10 ##########################################################
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%

***This is happening on 2 different stacks that are both 2960 based.

Here are the STP invalid frames:

sh controllers cpu-interface

cpu-queue-frames retrieved dropped invalid hol-block stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc 21168866 0 0 0 0
stp 4694699 0 626394 0 0

Any other ideas? My concern is that we are seeing STP errors and high utilization. This util is consistent and doesn't show any obvious traffic concern.

Thanks.
D

rochey2009

Have you tried breaking the redundant links to see if the problem goes away?

ncomper

ASKER

To be honest we haven't. Cisco do recommend to physically disconnect any blocked links as a diagnostic. I will make arrangements for this and let you know how we get on.

ncomper

ASKER

Also from the processes list on both of our stacks only I am seeing high utilization from 'HULC LED Process'.

ASKER CERTIFIED SOLUTION

ncomper

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ncomper

ASKER

No solution found.

Partial solution was updating the IOS on our access switches.