Terminal Server High CPU / RAM utilization - How to pinpoint and rectify cause?

I have a terminal server that was built during a crisis, and had a lot of applications thrown on it willy nilly. Presently, there are 36 users configured on it, with average active sessions sitting somewhere around 25.

Normally, CPU and RAM seem to sweat but not crap out. Recently, however, we've had a series of sustained spikes that appear to be coming from, among other apps, Firefox. When I open Windows Task Manager on this server using my Domain Admin profile, the usual finding is many instances of Firefox eating RAM. Today, however, the server nearly came to a halt from what initially looked like an overloaded CPU. The problem with that is, when I sort processes for CPU, they don't add up even close to 100% utilization. This is with "Show processes from all users" selected.

Question: How do I properly use/interpret the performance and processes tab to identify problematic applications? Should I be using something else? What other terminal server crisis strategies can you share? :)

I just logged off disconnected users and sent a message to active users indicating they should use only essential applications. I also tweaked McAfee a bit. The situation has improved, but I would like find a way to pinpoint this sort of thing quickly and conclusively, as well just know how others handle these sorts of issues.

Any input is much appreciated!
carnescAsked:
Who is Participating?
 
zelron22Commented:
If it's a multi-cpu machine, what can happen is one CPU gets overloaded if the processes aren't balanced, and even though the server is running at 50%, one CPU is at 100% and the other is idling.

http://msdn.microsoft.com/en-us/library/aa383520(VS.85).aspx
0
 
carnescAuthor Commented:
How can I satisfy myself (and my boss ) that this is occurring? This is a blade system, running two, four-core Intel processors. We had to throw this server together during an emergency build, and against my will some older (2K) apps were installed, as were apps designed more for desktop installs.

Here's another specific question I have: Do you have any experience with Firefox giving grief within a Terminal Server environment? In our case, it definitely seems to be a recurring offender with respect to RAM. Most unexpectedly, it routinely displays nearly twice the RAM allocation per process as IE. When I sort for RAM utilization, I see Firefox, Firefox, Firefox, ...all at the very top of the list.
0
 
zelron22Commented:
Well, if you look at the task manager when the server is having the problem, is one of the four CPUs running at 100% or close to it?

Make sure McAfee isn't scanning network drives.

Nope.  I support IE in my environment and people are welcome to use Firefox on their workstations, but I don't support it.  It may be a great product, but I can only support so many apps, so if they have a problem with it, they're on their own.  If they have a problem with something else, I may uninstall it to troubleshoot.
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

 
carnescAuthor Commented:
The Performance pane of Task Manager was showing all cores maxed out across the board, but the processes pane, when sorted for CPU, did not add up. Lots of RAM being chewed up by Firefox, but no real CPU usage to speak of.

Also, I definitely have McAfee configured to not scan network drives. It's "mini-firewall" feature is enabled for "Maximum Protection", but I've entered exclusions for major applications. That aspect of the installation has been working OK for about a month now.

The only spikes we typically see seem to revolve around firefox, but given the disparity between Performance and Process tab CPU reporting, I am not sure how much to make of the Firefox readings.
0
 
zelron22Commented:
How much available physical RAM do you have when this happens?  
I think I would disable the firewall for a while to test (assuming you don't have this TS wide open to the Internet).
0
 
carnescAuthor Commented:
There is 32GB in the blade, but I can't say that I have taken note of the exact amount available during one of these CPU spikes. One of my coworkers primarily monitored this server until he went on vacation this week. :) When my phone started lighting up and I saw the server suddenly teetering between 95-100% CPU utilization, my available deductive logic pretty much stopped at the big, graphical flatline. :)

I do remember that the RAM was not maxed out and that, among that portion allocated, many instances of Firefox were fighting each other for top consumer.
0
 
zelron22Commented:
Are they using firefox to browse the internet or for an application?  If they're using it for browsing the internet, I think they'd be better of browsing from their own machine (unless of course it's a DOS based thin client or workstation or something).

Here's a link talking about some older issues with Firefox which still may apply.
http://kb.mozillazine.org/Firefox_CPU_usage
0
 
carnescAuthor Commented:
These folks are on Sun Ray thin clients, and the usage it almost universally for intranet and Web-based monitoring/administrative pages.

Well, we definitely found the problem. It happened again today and with more Terminal Servers. This time, however, I actually did see CPU utilization spikes by process, not just within the performance tab. Again, it was Firefox. What is interesting is that the problem was isolated to using Firefox to view our 'Google Apps for Enterprise' pages. We use Google as our collaboration outsource.

I identified our top ten offenders and observed their sessions. The very first case involved a guy with ten Firefox tabs open, all intranet plus one Google Mail tab. I was monitoring his resource utilization as he began closing tabs one by one. The moment the Google Mail tab closed, the CPU dropped straight to zero. Identical behavior was observed with every other user.

The temporary fix was to have everyone keep open a coupe Internet Explorer tabs just for Google applications and continue using Firefox for everything else. Incidentally, similar spike were also observed with IE, but these processes promptly released the CPU as expected.

I am unsure whether the problem results from Firefox alone; a plug-in alone; Google code; or any combination of the three. That's definitely the ballpark though. We are using the most recent version of Firefox, achieved by way of repeated upgrades. I am reinstalling this app tonight from scratch to see if the problem disappears. If not, everyone knows the workaround.

Have not yet read your link, but I will momentarily. I still wonder why I was not at first able to observe CPU spikes by process consistent with that observed in the performance tab.
0
 
zelron22Commented:
It's a "feature".  Great work.
0
 
carnescAuthor Commented:
Thanks for the assistance! All's well the ends well.

Maybe I just wasn't initially watching the process tab carefully enough. As I recall, the first time this happened, the machine would barely respond to anything. I may have thought it sorted by CPU, but in really it may not have completed the task. Who knows. I definitely saw it the second time, so I think I am back to trusting Task Manager. :)
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.