Slow Performance IBM x3550 M2 Vmware VSphere 4.0.0 Update 1


We are a Small Business

I have a speed issue with an IBM  x3550 M2 VSphere 4.0.0 Update 1
1x Xeon 5530
12 gig RAM
1 IBM MR10i
4 Disc (RAID 5) 300 gb SAS 2 10k
2 Disc (RAID 0) to test. 300 gb SAS 2 10k
6 NIC (1 SC, 1 Failover, 4 active)

1 HP Procurve Switch 1400-24G

3 VMs
1 Linux Centos 5.4 -  MySQL, 50 meg DB
1 Web Server - 2003 Server IIS (to display our website)
1 SQL Server 2000 Sp4 - 2003 Server  (8.5 gig DB)
Disc configuration: LSI Logic Parrallel
1 vswitch (3 uplinks 1 gig)

 The problem started from day 2 with slow response time from the SQL Server VM during the day; 5 - 6 users within our ERP software installed on a physical server box (Terminal Server). The ERP query SQL Server.

When I copy a 8 gig files from a VM to a physical server (a new machine, 1 gig NIC, RAID 5...) it take around 30 min.

I import a Virtual Machine Workstation with vConvertor (10 gig file) to test the speed it take around 45 min.

Tests done so far :

I had a 3Com baseline 1 gbs switch, I changed it for a HP Procurve 1400-24G brand new (I know it's basic, but it's a small company 15 users). no gain.

I created a vswitch to assign only "One Physical NIC" to the SQL VM no gain. I upgrade the vNIC to Vmnet3 no gain.

All the 3 Vms were running over the RAID 5, I try to improve the speed I installed 2 news 300 Gbs and create a RAID 0 and I move the SQL VM over. I didn't get any better result !!

The Disc latency average is around 10 - 20 ms.

Can the Mr10i was my bottleneck ?

Any suggestion to improve the speed or figure out the problem?

Thanks for your help.

Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

If you are doing any significant amount of I/O, then no way will a 4 disk RAID5 SAS array meet these needs.   Latency is a clue.   Need to add more disk drives, preferably RAID10 for database, and a pair of disks in RAID1 for O/S.   if content is  dynamically generated and you are logging web hits and such then maybe even a separate RAID1 for that as well. No matter what, you need more disk drives.  

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
bmdgiAuthor Commented:
I already did that, I added 2 new 300 gb 10k SAS2 drive in RAID0 and I get no significant speed.

That's why i'm suspecting the RAID controller can be the bottleneck ?

But how did you configure the storage?  What is the native block size of the RAID? What is the NTFS allocation size?  What is the native I/O size of VMWare?   Does it all match? Doubtful.  You could very well have highly misconfigured storage.  You absolutely have grossly misconfigured storage if you took defaults.   For example SQL server does I/O in 64KB chunks.  Default on NTFS is 4KB.   So when SQL does a write this gets translated into 16 x 4KB writes (but NTFS will somewhat aggregate this) .. but then that will do a bunch of much larger writes on VMWARE.

So go down this path of figuring out the block size that each of these 3 components are configured to read or write at a time, and make them match.  You probably send a minimum of 128KB of data to disk whenever your machine wants to write 4KB ... or perhaps it is much worse, you could be writing up to 16x128KB for every 64KB SQL write in worst case scenario
Powerful Yet Easy-to-Use Network Monitoring

Identify excessive bandwidth utilization or unexpected application traffic with SolarWinds Bandwidth Analyzer Pack.

bmdgiAuthor Commented:
I will need to test the configuration as you suggest. I get the default configuration from the controller and I convert the 3 VM from physical IBM server So If I understand, I should change the configuration of the controller to 128k size for my volume should this help ?

Everything needs to match. Make sure NTFS is using aligned I/Os also.  google "partition aligned I/O NTFS" to see how it is done.   You can't change ANY of these settings w/o a full backup and reformatting operation.  None of it is on-the-fly.  Sorry
bmdgiAuthor Commented:
Right now everything is formatted by default, NTFS drive and Linux EXT3 partition and the Raid controller

To start correctly
What is the configuration I should do in the RAID controller : 128K RAID 10
And reinstall ESX4. At this point even if my VM is not formatted with 128K should I get any significant result ?

What is the native blocking for VMware? I do not remember. You need to look that up because I don't think it is flexible.  the RAID LUN needs to match. Start from there.
bmdgiAuthor Commented:
I recreate the setup in RAID10 with 1 meg as the block size because VMFS is 1 meg.

The speed is really better.

Curious ... what was the before & after speed?
bmdgiAuthor Commented:
If I copied a large files (inside the VM (2003) , from drive d to drive d) 9 gig, before : around 45 min.
Now : 6 - 8 min.

I have a better sustain rates at this point.

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.