Solved

High CPU Utilization on Cisco 2960C Stack

Posted on 2011-09-27
8
2,943 Views
Last Modified: 2012-05-12
Hi,

I am seeing this on both of my 2960 stacks:

CPU utilization for five seconds: 40%/13%; one minute: 56%; five minutes: 46%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
133      912176     88773      10275 16.29% 16.30% 16.41%   0 Hulc LED Process
COLMDPS01#sh processes cpu | exclude 0.00%
CPU utilization for five seconds: 32%/8%; one minute: 40%; five minutes: 44%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  11        7076     15291        462  0.19%  0.14%  0.13%   0 ARP Input
  43        2175       948       2294  0.09%  0.03%  0.03%   0 Compute load avg
  63        3248     33354         97  0.19%  0.06%  0.05%   0 RedEarth Tx Mana
 103       46470     10011       4641  0.59%  0.81%  0.80%   0 hpm counter proc
 104        3462     19965        173  0.19%  0.08%  0.08%   0 HRPC pm-counters
133      927885     90266      10279 16.65% 16.33% 16.39%   0 Hulc LED Process

IOS Version:

Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
*    1 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M
     2 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M
     3 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M

I have done some troubleshooting at Cisco recommendation and the only thing I can find that may be a concern is this:

sh controllers cpu-interface
<snip>
cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               171278     0          0          0          0
stp               35627      0          4756       0          0

Initally I can't see any problem with RSTP across the network.

Does anyone have any suggestions?

Thanks.
D

0
Comment
Question by:ncomper
  • 5
  • 2
8 Comments
 
LVL 26

Expert Comment

by:Soulja
ID: 36709694
I would double check your connections, making sure there are not loops. Make sure bpdu guard is enabled on all access ports. Make sure you have your desired root for each of your vlans.
0
 
LVL 17

Expert Comment

by:rochey2009
ID: 36713074
How many vlans do you have?
0
 
LVL 5

Author Comment

by:ncomper
ID: 36909338
There are redundant links in the switched network.

STP has blocked the correct ports where there is redundancy.

There are no ports in the 'err-disabled' state.

The root bridge is set correctly for each vlan. (We are running PVST)

BDUGuard is running on all access ports. (Combined with portfast)

Last 60 minutes:

100
 90                                                  *
 80  ********** ******************************** ***#*********
 70  *########################################################
 60 **########################################################
 50 **########################################################
 40 ##########################################################
 30 ##########################################################
 20 ##########################################################
 10 ##########################################################
   0....5....1....1....2....2....3....3....4....4....5....5....
             0    5    0    5    0    5    0    5    0    5
               CPU% per minute (last 60 minutes)
              * = maximum CPU%   # = average CPU%


***This is happening on 2 different stacks that are both 2960 based.

Here are the STP invalid frames:

sh controllers cpu-interface

cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               21168866   0          0          0          0
stp               4694699    0          626394     0          0


Any other ideas? My concern is that we are seeing STP errors and high utilization. This util is consistent and doesn't show any obvious traffic concern.

Thanks.
D
0
 
LVL 17

Expert Comment

by:rochey2009
ID: 36912451
Have you tried breaking the redundant links to see if the problem goes away?
0
Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

 
LVL 5

Author Comment

by:ncomper
ID: 36915342
To be honest we haven't. Cisco do recommend to physically disconnect any blocked links as a diagnostic. I will make arrangements for this and let you know how we get on.
0
 
LVL 5

Author Comment

by:ncomper
ID: 36915666
Also from the processes list on both of our stacks only I am seeing high utilization from 'HULC LED Process'.
0
 
LVL 5

Accepted Solution

by:
ncomper earned 0 total points
ID: 36962218
Can anyone please confirm what we can do with these STP errors:

sh controllers cpu-interface
<snip>
cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               171278     0          0          0          0
stp               35627      0          4756       0          0
0
 
LVL 5

Author Closing Comment

by:ncomper
ID: 37010485
No solution found.

Partial solution was updating the IOS on our access switches.
0

Featured Post

Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Outlook keeps opened file locked 2 53
How computer Arp Table gets populated. 21 86
Interface VLAN dependencies 6 30
cmd: can't connect using netsh command 14 30
Meet the world's only “Transparent Cloud™” from Superb Internet Corporation. Now, you can experience firsthand a cloud platform that consistently outperforms Amazon Web Services (AWS), IBM’s Softlayer, and Microsoft’s Azure when it comes to CPU and …
For many of us, the  holiday season kindles the natural urge to give back to our friends, family members and communities. While it's easy for friends to notice the impact of such deeds, understanding the contributions of businesses and enterprises i…
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

919 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now