?
Solved

High CPU Utilization on Cisco 2960C Stack

Posted on 2011-09-27
8
Medium Priority
?
3,205 Views
Last Modified: 2012-05-12
Hi,

I am seeing this on both of my 2960 stacks:

CPU utilization for five seconds: 40%/13%; one minute: 56%; five minutes: 46%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
133      912176     88773      10275 16.29% 16.30% 16.41%   0 Hulc LED Process
COLMDPS01#sh processes cpu | exclude 0.00%
CPU utilization for five seconds: 32%/8%; one minute: 40%; five minutes: 44%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  11        7076     15291        462  0.19%  0.14%  0.13%   0 ARP Input
  43        2175       948       2294  0.09%  0.03%  0.03%   0 Compute load avg
  63        3248     33354         97  0.19%  0.06%  0.05%   0 RedEarth Tx Mana
 103       46470     10011       4641  0.59%  0.81%  0.80%   0 hpm counter proc
 104        3462     19965        173  0.19%  0.08%  0.08%   0 HRPC pm-counters
133      927885     90266      10279 16.65% 16.33% 16.39%   0 Hulc LED Process

IOS Version:

Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
*    1 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M
     2 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M
     3 52    WS-C2960S-48FPS-L  12.2(53)SE2           C2960S-UNIVERSALK9-M

I have done some troubleshooting at Cisco recommendation and the only thing I can find that may be a concern is this:

sh controllers cpu-interface
<snip>
cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               171278     0          0          0          0
stp               35627      0          4756       0          0

Initally I can't see any problem with RSTP across the network.

Does anyone have any suggestions?

Thanks.
D

0
Comment
Question by:ncomper
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 2
8 Comments
 
LVL 26

Expert Comment

by:Soulja
ID: 36709694
I would double check your connections, making sure there are not loops. Make sure bpdu guard is enabled on all access ports. Make sure you have your desired root for each of your vlans.
0
 
LVL 17

Expert Comment

by:rochey2009
ID: 36713074
How many vlans do you have?
0
 
LVL 5

Author Comment

by:ncomper
ID: 36909338
There are redundant links in the switched network.

STP has blocked the correct ports where there is redundancy.

There are no ports in the 'err-disabled' state.

The root bridge is set correctly for each vlan. (We are running PVST)

BDUGuard is running on all access ports. (Combined with portfast)

Last 60 minutes:

100
 90                                                  *
 80  ********** ******************************** ***#*********
 70  *########################################################
 60 **########################################################
 50 **########################################################
 40 ##########################################################
 30 ##########################################################
 20 ##########################################################
 10 ##########################################################
   0....5....1....1....2....2....3....3....4....4....5....5....
             0    5    0    5    0    5    0    5    0    5
               CPU% per minute (last 60 minutes)
              * = maximum CPU%   # = average CPU%


***This is happening on 2 different stacks that are both 2960 based.

Here are the STP invalid frames:

sh controllers cpu-interface

cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               21168866   0          0          0          0
stp               4694699    0          626394     0          0


Any other ideas? My concern is that we are seeing STP errors and high utilization. This util is consistent and doesn't show any obvious traffic concern.

Thanks.
D
0
NFR key for Veeam Agent for Linux

Veeam is happy to provide a free NFR license for one year.  It allows for the non‑production use and valid for five workstations and two servers. Veeam Agent for Linux is a simple backup tool for your Linux installations, both on‑premises and in the public cloud.

 
LVL 17

Expert Comment

by:rochey2009
ID: 36912451
Have you tried breaking the redundant links to see if the problem goes away?
0
 
LVL 5

Author Comment

by:ncomper
ID: 36915342
To be honest we haven't. Cisco do recommend to physically disconnect any blocked links as a diagnostic. I will make arrangements for this and let you know how we get on.
0
 
LVL 5

Author Comment

by:ncomper
ID: 36915666
Also from the processes list on both of our stacks only I am seeing high utilization from 'HULC LED Process'.
0
 
LVL 5

Accepted Solution

by:
ncomper earned 0 total points
ID: 36962218
Can anyone please confirm what we can do with these STP errors:

sh controllers cpu-interface
<snip>
cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc               171278     0          0          0          0
stp               35627      0          4756       0          0
0
 
LVL 5

Author Closing Comment

by:ncomper
ID: 37010485
No solution found.

Partial solution was updating the IOS on our access switches.
0

Featured Post

Get real performance insights from real users

Key features:
- Total Pages Views and Load times
- Top Pages Viewed and Load Times
- Real Time Site Page Build Performance
- Users’ Browser and Platform Performance
- Geographic User Breakdown
- And more

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article is a collection of issues that people face from time to time and possible solutions to those issues. I hope you enjoy reading it.
This article explains the fundamentals of industrial networking which ultimately is the backbone network which is providing communications for process devices like robots and other not so interesting stuff.
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
In this video we outline the Physical Segments view of NetCrunch network monitor. By following this brief how-to video, you will be able to learn how NetCrunch visualizes your network, how granular is the information collected, as well as where to f…
Suggested Courses
Course of the Month10 days, 5 hours left to enroll

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question