Solved

High CPU Utilization on Cisco 2960C Stack

Posted on 2011-09-27
8
2,988 Views
Last Modified: 2012-05-12
Hi,

I am seeing this on both of my 2960 stacks:

CPU utilization for five seconds: 40%/13%; one minute: 56%; five minutes: 46%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
133      912176     88773      10275 16.29% 16.30% 16.41%   0 Hulc LED Process
COLMDPS01#sh processes cpu | exclude 0.00%
CPU utilization for five seconds: 32%/8%; one minute: 40%; five minutes: 44%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  11        7076     15291        462  0.19%  0.14%  0.13%   0 ARP Input
  43        2175       948       2294  0.09%  0.03%  0.03%   0 Compute load avg
  63        3248     33354         97  0.19%  0.06%  0.05%   0 RedEarth Tx Mana
 103       46470     10011       4641  0.59%  0.81%  0.80%   0 hpm counter proc
 104        3462     19965        173  0.19%  0.08%  0.08%   0 HRPC pm-counters
133      927885     90266      10279 16.65% 16.33% 16.39%   0 Hulc LED Process

IOS Version:

Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
*    1 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M
     2 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M
     3 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M

I have done some troubleshooting at Cisco recommendation and the only thing I can find that may be a concern is this:

sh controllers cpu-interface
<snip>
cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               171278     0          0          0          0
stp               35627      0          4756       0          0

Initally I can't see any problem with RSTP across the network.

Does anyone have any suggestions?

Thanks.
D

0
Comment
Question by:ncomper
  • 5
  • 2
8 Comments
 
LVL 26

Expert Comment

by:Soulja
ID: 36709694
I would double check your connections, making sure there are not loops. Make sure bpdu guard is enabled on all access ports. Make sure you have your desired root for each of your vlans.
0
 
LVL 17

Expert Comment

by:rochey2009
ID: 36713074
How many vlans do you have?
0
 
LVL 5

Author Comment

by:ncomper
ID: 36909338
There are redundant links in the switched network.

STP has blocked the correct ports where there is redundancy.

There are no ports in the 'err-disabled' state.

The root bridge is set correctly for each vlan. (We are running PVST)

BDUGuard is running on all access ports. (Combined with portfast)

Last 60 minutes:

100
 90                                                  *
 80  ********** ******************************** ***#*********
 70  *########################################################
 60 **########################################################
 50 **########################################################
 40 ##########################################################
 30 ##########################################################
 20 ##########################################################
 10 ##########################################################
   0....5....1....1....2....2....3....3....4....4....5....5....
             0    5    0    5    0    5    0    5    0    5
               CPU% per minute (last 60 minutes)
              * = maximum CPU%   # = average CPU%


***This is happening on 2 different stacks that are both 2960 based.

Here are the STP invalid frames:

sh controllers cpu-interface

cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               21168866   0          0          0          0
stp               4694699    0          626394     0          0


Any other ideas? My concern is that we are seeing STP errors and high utilization. This util is consistent and doesn't show any obvious traffic concern.

Thanks.
D
0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 
LVL 17

Expert Comment

by:rochey2009
ID: 36912451
Have you tried breaking the redundant links to see if the problem goes away?
0
 
LVL 5

Author Comment

by:ncomper
ID: 36915342
To be honest we haven't. Cisco do recommend to physically disconnect any blocked links as a diagnostic. I will make arrangements for this and let you know how we get on.
0
 
LVL 5

Author Comment

by:ncomper
ID: 36915666
Also from the processes list on both of our stacks only I am seeing high utilization from 'HULC LED Process'.
0
 
LVL 5

Accepted Solution

by:
ncomper earned 0 total points
ID: 36962218
Can anyone please confirm what we can do with these STP errors:

sh controllers cpu-interface
<snip>
cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               171278     0          0          0          0
stp               35627      0          4756       0          0
0
 
LVL 5

Author Closing Comment

by:ncomper
ID: 37010485
No solution found.

Partial solution was updating the IOS on our access switches.
0

Featured Post

Live: Real-Time Solutions, Start Here

Receive instant 1:1 support from technology experts, using our real-time conversation and whiteboard interface. Your first 5 minutes are always free.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Even if you have implemented a Mobile Device Management solution company wide, it is a good idea to make sure you are taking into account all of the major risks to your electronic protected health information (ePHI).
This article will inform Clients about common and important expectations from the freelancers (Experts) who are looking at your Gig.
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…

816 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now