100% CPU utilization with high traffic and spiky CPU with low traffic

Hello,
We have a .Net 2.0 site which is based on Ektron the CMS and while the site operates and IIS never crashes we are unable to keep up with traffic due to high CPU usage. It seems there is a tipping point when we get near 100 users or more on the site.

Using process explorer I see that both my Xeon 3.0ghz CPU's will be pegged at 100% on w3wp.exe which is the IIS worker process. Inside that process there are .Net dll calls (mscorwks.dll!CreateApplicationContext) which are taking 20-40% of the CPU. I see the same behavior with low traffic but it's enough that the boxes can handle it.

There are 3 web servers behind a load balancer and too much traffic during elevated times so I need to find a fix. I believe it's a code issue in .Net as I have about 60 other webservers with similar .Net MSSQL setup and no issues. Some of those servers have 100 times the load with no issues at all.

As a test we installed the new .Net 2.0 Service Pack 1 which was released on January 23rd to see if that might fix it as it had several rollup patches and hotfixes which updated mscorwks.dll. This did not change the spiking CPU however instead of  "mscorwks.dll!CreateApplicationContext" we see "mscorwks.dll!ClrCreateManagedInstance".

We have been running perfmon and SQL traces for days and everything points back to .Net itself being the issue. The hardware, network, and DB side of things appears to be stable and smooth. A general thought might be to add more hardware but sincerely thats not an option and I have other servers with literally 100 times the load with no problem running other sides.

Does anyone have any idea's on what this might be?
2drewsAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

2drewsAuthor Commented:
Hello,
We have found a solution to this on our own and the answer was both code and web server changes were needed to get optimal performance.

In IIS Microsoft had us change the deftaul thread count and bump it all the way to 200 per worker process. This helped it quite a bit.

Addtionally we broke up the application into functional elements and tested them individually. The dynamic navigation menu was the major holdup as well. This was fixed by tweaking the stored procedures on the application.

I didn't expect anyone to know application specifics but more of a direction to head in. I am taking the time to comment in case this helps anyone else with the same issue in the future.
0
ThorinCommented:
2drews,

I am having the same problem as you describe.  Can you provide any more details on your work arounds?  This is a terrible problem!  Thanks!

-Thorin
0
Computer101Commented:
PAQed with points refunded (500)

Computer101
EE Admin
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft IIS Web Server

From novice to tech pro — start learning today.