Exchange Server Vol that holds Trans Logs has an extermely high Avg. Disk Queue Length

Users are experiencing extreme latencies within Outlook. This has been getting progressively worse.
I ran some Performance Monitoring on the server and found that the volume (D:\) has a very high Avg Disk Queue Length.
over a 1:40 minute duration
Low: 1.126
High: 296.741
Avg: 17.352
This server is setup as:
C:\ is the OS (RAID1)
D:\ Trans action and Exchange program directory (RAID1)
E:\ & F:\  Information stores RAID5  

I am not seeing high Avg Disk Queue's, Reads or Writes on the E: & F: drive.  

Would I be better off creating a new ARRAY RAID5 for the Trans Logs?
LVL 27
yo_beeDirector of Information TechnologyAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

valmaticCommented:
I see the raid setup now. RAID 1 not going to gain you any performance on the trans logs. Raid 5 would be better if you had the disks. Are you backing it up where the logs are getting flushed every night? If not the logs can get huge, what size are you seeing on that disk for the logs.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
yo_beeDirector of Information TechnologyAuthor Commented:
Full backups once a week daily incremental (All logs are flushed).
I do have the disks to build a RAID5 for the Log Vol.
So you would recommend that I create this ARRAY and move the Trans Logs to the this location.
0
Gareth GudgerSolution ArchitectCommented:
Those or some high spikes.

Do you have antivirus loaded onto the server? If so, have you excluded all the necessary file locations and Exchange processes?

This is Microsoft's document on what you should exclude from all antivirus scans (whether on demand, real time or scheduled). http://technet.microsoft.com/en-us/library/bb332342(v=exchg.150).aspx

My rule of thumb is to always exclude the Databases, Log Files and the Exchange Install Directory at a minimum. So in your case you may want to completely exclude all of D, E and F.

What about backup windows? Have you confirmed backups are not running over during these times of spikes and poor performance?

Shadow Copies are disabled on those volumes right?

I am assuming this is a physical server. But if virtual, any kind of volume snapping or virtual snap could be causes performance related issues.
0
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

yo_beeDirector of Information TechnologyAuthor Commented:
I am doing some more analysis on this server.  I am collecting Avg Disk Transfer/sec per vol, user Counts, Avg Disk Read/sec Avg Disk Write/sec, Avg Disk Queue Length

It is a physical one and old one as well.  I am still running Exchange 2003.

Anti-Virus has been removed for troubleshooting.  Still avg 30% CPU usage for Store.exe.

I am in the beginning stages of putting together the right setup for my 2010.

I need to make sure that I have it properly spec out.  
It will be Virtual.  I want to make sure my SAN is prepared for this addition.

I will post the results later.
0
Gareth GudgerSolution ArchitectCommented:
Hey yo bee,

Glad you are getting off Exchange 2003. One of the biggest benefits of Exchange 2010 is that the database I/O is 90% less than Exchange 2003! Crazy huh?

If you are looking for a step-by-step guide on how to go from 2003 to 2010 then check out this series. Lots of screenshots! :)
http://supertekboy.com/2014/03/31/migrating-exchange-2003-2010-part/
http://supertekboy.com/2014/04/02/migrating-exchange-2003-2010-part-ii/
http://supertekboy.com/2014/04/07/migrating-exchange-2003-2010-part-iii/
http://supertekboy.com/2014/04/14/migrating-exchange-2003-2010-part-iv/
http://supertekboy.com/2014/04/19/migrating-exchange-2003-2010-part-v/
0
yo_beeDirector of Information TechnologyAuthor Commented:
Thanks Gareth.  I will look at these.

My biggest concern right now when migrating over is the SAN DISK configurations.  I will be reaching out to NETAPP to have one of their engineers to help explain some things.
0
Gareth GudgerSolution ArchitectCommented:
What kind of concerns do you have? The I/O on 2010 DBs is 90% less than 2003. This is where Microsoft comes in and says you can put Exchange on JBOD or 7200k RPM SATA drives. :)
0
yo_beeDirector of Information TechnologyAuthor Commented:
My concern is my current SAN configuration and if I add the new Exchange 2010 into the mix will things get worse?
Right now users are experiencing latency in multiple areas.  CIFS shares from the SAN, I see some slowness on Server (VM).  We have 5 SQL servers running in the VM infrastructure.  The SQL DB's are attached to the VM's as LUN stored on the 15K SAS disks.
 
We are using a NETAPP Data on Tap model.
We have two type of disks.  One Shelf with SAS 15K RPM and the other two shelfs SATA 7200 RPM.  
All of our VM hosted by VMware and the VMDK's are stored on the SAS 15K Aggregate.
0
Gareth GudgerSolution ArchitectCommented:
Sounds like you need to talk to your SAN guys. Especially if you have significant I/O on there already.

Here is a great article on storage supportability for Exchange 2010. Make sure that you present SAN storage as block level and not file level storage. For example, iSCSI LUNs.
http://technet.microsoft.com/en-us/library/ee832792(v=exchg.150).aspx
0
yo_beeDirector of Information TechnologyAuthor Commented:
Thanks Gareth
0
yo_beeDirector of Information TechnologyAuthor Commented:
I divided the points between both of you.
I was able to add 4 more disks in a RAID 0 + 1 and moved the Trans Logs from the D:\Programs files\exch\ to g:\Program\files\exch\<IS Log> . This change seems to have a much better response between the clients and server. The latency seems to have dropped and users are not seeing the Waiting For Exchange Balloon.  I have not collected any matrix since the change, but I have to think it going to be much lower.

I am still see avg 30% CPU usage by Store.exe, but this server has 3 Info Stores, 15 DB's that comprise 1.78 TB worth of data.  This does not seem to be a concern that I had initially.

I have to started to research the recommended link as of yet, but I am confident that they have good information.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Outlook

From novice to tech pro — start learning today.