Solved

High CPU Utilization on Cisco 2960C Stack

Posted on 2011-09-27
8
3,138 Views
Last Modified: 2012-05-12
Hi,

I am seeing this on both of my 2960 stacks:

CPU utilization for five seconds: 40%/13%; one minute: 56%; five minutes: 46%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
133      912176     88773      10275 16.29% 16.30% 16.41%   0 Hulc LED Process
COLMDPS01#sh processes cpu | exclude 0.00%
CPU utilization for five seconds: 32%/8%; one minute: 40%; five minutes: 44%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  11        7076     15291        462  0.19%  0.14%  0.13%   0 ARP Input
  43        2175       948       2294  0.09%  0.03%  0.03%   0 Compute load avg
  63        3248     33354         97  0.19%  0.06%  0.05%   0 RedEarth Tx Mana
 103       46470     10011       4641  0.59%  0.81%  0.80%   0 hpm counter proc
 104        3462     19965        173  0.19%  0.08%  0.08%   0 HRPC pm-counters
133      927885     90266      10279 16.65% 16.33% 16.39%   0 Hulc LED Process

IOS Version:

Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
*    1 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M
     2 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M
     3 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M

I have done some troubleshooting at Cisco recommendation and the only thing I can find that may be a concern is this:

sh controllers cpu-interface
<snip>
cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               171278     0          0          0          0
stp               35627      0          4756       0          0

Initally I can't see any problem with RSTP across the network.

Does anyone have any suggestions?

Thanks.
D

0
Comment
Question by:ncomper
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 2
8 Comments
 
LVL 26

Expert Comment

by:Soulja
ID: 36709694
I would double check your connections, making sure there are not loops. Make sure bpdu guard is enabled on all access ports. Make sure you have your desired root for each of your vlans.
0
 
LVL 17

Expert Comment

by:rochey2009
ID: 36713074
How many vlans do you have?
0
 
LVL 5

Author Comment

by:ncomper
ID: 36909338
There are redundant links in the switched network.

STP has blocked the correct ports where there is redundancy.

There are no ports in the 'err-disabled' state.

The root bridge is set correctly for each vlan. (We are running PVST)

BDUGuard is running on all access ports. (Combined with portfast)

Last 60 minutes:

100
 90                                                  *
 80  ********** ******************************** ***#*********
 70  *########################################################
 60 **########################################################
 50 **########################################################
 40 ##########################################################
 30 ##########################################################
 20 ##########################################################
 10 ##########################################################
   0....5....1....1....2....2....3....3....4....4....5....5....
             0    5    0    5    0    5    0    5    0    5
               CPU% per minute (last 60 minutes)
              * = maximum CPU%   # = average CPU%


***This is happening on 2 different stacks that are both 2960 based.

Here are the STP invalid frames:

sh controllers cpu-interface

cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               21168866   0          0          0          0
stp               4694699    0          626394     0          0


Any other ideas? My concern is that we are seeing STP errors and high utilization. This util is consistent and doesn't show any obvious traffic concern.

Thanks.
D
0
Manage your data center from practically anywhere

The KN8164V features HD resolution of 1920 x 1200, FIPS 140-2 with level 1 security standards and virtual media transmissions at twice the speed. Built for reliability, the KN series provides local console and remote over IP access, ensuring 24/7 availability to all servers.

 
LVL 17

Expert Comment

by:rochey2009
ID: 36912451
Have you tried breaking the redundant links to see if the problem goes away?
0
 
LVL 5

Author Comment

by:ncomper
ID: 36915342
To be honest we haven't. Cisco do recommend to physically disconnect any blocked links as a diagnostic. I will make arrangements for this and let you know how we get on.
0
 
LVL 5

Author Comment

by:ncomper
ID: 36915666
Also from the processes list on both of our stacks only I am seeing high utilization from 'HULC LED Process'.
0
 
LVL 5

Accepted Solution

by:
ncomper earned 0 total points
ID: 36962218
Can anyone please confirm what we can do with these STP errors:

sh controllers cpu-interface
<snip>
cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               171278     0          0          0          0
stp               35627      0          4756       0          0
0
 
LVL 5

Author Closing Comment

by:ncomper
ID: 37010485
No solution found.

Partial solution was updating the IOS on our access switches.
0

Featured Post

NEW Veeam Agent for Microsoft Windows

Backup and recover physical and cloud-based servers and workstations, as well as endpoint devices that belong to remote users. Avoid downtime and data loss quickly and easily for Windows-based physical or public cloud-based workloads!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article, I am going to show you how to simulate a multi-site Lab environment on a single Hyper-V host. I use this method successfully in my own lab to simulate three fully routed global AD Sites on a Windows 10 Hyper-V host.
Most of the applications these days are on Cloud. Cloud is ubiquitous with many service providers in the market. Since it has many benefits such as cost reduction, software updates, remote access, disaster recovery and much more.
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
Monitoring a network: why having a policy is the best policy? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the enormous benefits of having a policy-based approach when monitoring medium and large networks. Software utilized in this v…

717 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question