CPU utilization goes from 10% to 100% gradually over ~60mins of use

I have a CPU creep problem in my application and don't know how to troubleshoot it.  What happens is I will start up the app and turn on perfmon looking specifically at the CPU utilization of the application process and thread count.  The app will start off using about 10% CPU and 8-10 threads.  It gradually increases its load on the CPU and thread consumption over the course of about 60 mins after which time it is using 100% CPU and 60 threads.  I have to suspect that this is some how related to the multithreading but have no idea how to look at what part of the app is causing the CPU creep.  This is my first serious multithreaded app and i'm really at a loss.  I will award points for ideas about how to investigate the problem because it doesn't seem likely that anyone will be able to tell me what line of code is causing the problem... ;-)  Thanks!
LVL 1
stealth188Asked:
Who is Participating?
 
armoghanCommented:
0
 
sj_hicksCommented:
Process Explorer from http://www.sysinternals.com give lots of info about running processes such as threads, handles etc.  This might be helpful.
0
 
samittCommented:
For starters.. you could go to Perfmon and add counter ".NET CLRLocksAndThreads" for your application instance. Keep an eye on total no. of Contentions, Contention rate per second..
0
Cloud Class® Course: CompTIA Healthcare IT Tech

This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

 
stealth188Author Commented:
I was using spy++ and saw that there was contention happening, but I don't know how much is 'bad'.  How would I determine that such and such code is producing the contention?  Can you offer any advice?
0
 
samittCommented:
It's never very easy to debug... the way I would approach it is to break the app into smaller pieces.. write out log / trace messages at each thread start / stop  and probably with every other line of code..
In a well synchronized world any amount of thread contention should be avoided (but may not always be feasible).
0
 
samittCommented:
Are you user a 1 processor system? How is the GC implemented?
0
 
stealth188Author Commented:
I am using a 3.06 intel with HT.  Not sure how to answer GC question... If that gives you any indication ;)
0
 
stealth188Author Commented:
Took a look at the Contention Rate/sec.  It starts at about 1/sec and then after a while the couter increments with larger numbers.  It is at that point that the CPU starts to take off so it seems pretty reasonable to conclude that the two are related.  When you posted earlier you talked about writing trace messages when threads start/stop.  What information would be useful and should be included in the trace messages?  Contention is where multiple threads want to use a global variable correct?  there is only one situation where i use synclock in the app so I assume that has to be the source of the problem.  I tried to implement it using the smallest "footprint" possible.  Can you offer some advice on how to structure the synclocks?  Or possibly an alternative?  I am using Synclock to batch some SQL together and have included a code snipet below.  Thanks!

        SyncLock BatchSQL
            BatchSQL.Append(strSQL)
            BatchSQL.Append(";")
            BatchSQLCount += 1

            If BatchSQLCount >= 250 Then
                'If the SQL batch is big enough then copy the batch to a temp string variable and clear the global string variable
                SQLTemp = BatchSQL.ToString

                BatchSQL.Remove(0, BatchSQL.Length)
                BatchSQLCount = 0

                ExecuteBatch = True
            End If
        End SyncLock
0
 
gregoryyoungCommented:
that is not a long held lock and should not cause too much of an issue.

(form previous questions) there are other variables used within your app that are shared the async io on the socket is a good example.
0
 
stealth188Author Commented:
I took a look at the .net memory profiler.  I am new to the multithreading scene so it is taking me a while to understand what I am looking at.  What I think I have discovered is that a class I am creating to pass data to a thread is not being freed after the thread is finished.  So the question I think that I want to ask is how do I make sure that I have properly "disposed" of a class that has been used to send data to a thread after the thread has completed its job?
0
 
gregoryyoungCommented:
the object is a holder class and will be picked up during garbage collection. This is normal.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.