Solved

Seeking storage solution advice for VMWare ESXi rollout.

Posted on 2009-07-14
12
1,023 Views
Last Modified: 2012-05-07
The time has come to replace our servers and I am looking to take advantage of virtualization. I am having a hard time deciding whether or not I want to go with servers that have plenty of (yet finite) internal storage or go with small (1U) servers and a SAN. I've been toying with VMWare ESXi and so far, I love it. I noticed that it will allow me to add networked storage as datastores to store virtual machines.

Since I am looking at about 3 TB of total storage I was thinking that a SAN and many smaller disk may be the way to go for performance. However, I can get 3 TB in a single 2U server with SFF disk. Below is a list of all the servers I need to virtualize...

4 - Novell Netware 6.5 (to be upgraded to Suse Linux ES 10 SP2 with OES)
6 - Windows Server 2003 (already running under MS Virt Server 2005)
1 - Suse Linux ES 10 SP2

Combined current storage usage is about 1.5 TB so I want to double that across the board to 3 TB total. In order to plan for a spike in data storage I am leaning towards providing about 4TB. I serve about 100 users with mainly File/Print/E-Mail. We do have a few specialized applications that require the Windows boxes.

Does anyone have advice on whether to go with a couple large form factor machines with internal storage or a SAN? If you like SANs do you recommend iSCSI or Fibre?
0
Comment
Question by:jasonsfa98
12 Comments
 
LVL 24

Accepted Solution

by:
ryder0707 earned 150 total points
ID: 24855697
For SAN the backup is easier to manage especially if you decide to use VCB with your favourite backup software
Another advantage of have SAN as storage is you can share your storage among multiple hosts and use the esx advance feature, this is where the fun begins!
Both iscsi and fibre is good technology, everyone thinks that fibre is really fast(well it is)
iscsi can be fast too, depending on your design and budget, if you can allocate a pure dedicated IP network for your iscsi storage, it will be fast and cheap as well
and you are right, smaller disks for your VM are good for performance, in several enterprise customers i supported before, they normally present the LUN as required/needed, not really an estimation, so if you think you want 300GB now for 3 VMS
you can add a new 500GB datastore later for another few VMs
i always like to work with storage engineer, we may think that we know a lot about storage but when we deal with a dedicated storage guy, we'll learn a lot more :)



0
 
LVL 13

Assisted Solution

by:bleeuwen
bleeuwen earned 150 total points
ID: 24856855
I would go for ESX servers with shared storage (ryder0707 says, iscsi or FC does not matter, it depends on your needs and budget). The advantage with a san is that you can expand. You can now buy what you need and next year, if you need more, expand it.
If you need more processing power, add an ESX server to it. If more storage is needed, add disks or an expansion bay. Offcourse this will cost you money at first, but gives you a lot of flexibility later on. Advantages of more ESX servers with a san is that you can use vmotion. When the server needs service, you can vmotion your vm's off to an other esx and bring the esx down or do what ever you like.
11 virtuals normally can run on one hardware box (depending on your load offcourse), but then it's normally full and you do not have room for expansion. Begin with two ESX servers and a shared storage, expand later. If you need to watch your money, you can go for a single esx with local storage, but a few advantages are gone. When you have a hardware failure, all virtuals are down. If you want to expand and need a second ESX box, and you want to vmotion, you still need shared storage and rebuild your environment.
Watch performance though. IF you run 11 virtuals on the same disks (local storage), it needs IO. It could be that your local disks aren't that fast enough to provide you with enough io to give you the performance you need (disk bottleneck).
0
 
LVL 21

Assisted Solution

by:za_mkh
za_mkh earned 150 total points
ID: 24857201
In addition to the above, be aware that the max size for a LUN or datastore that ESX/i supports is 2TB. You can use Extents to increase this to a total of 64TB but think about that too when you do this.
We currently have a few Novell Servers running on ESX and they are great. At the moment, they the IPP/Zenworks servers and soon our Fileservers too.
Check your IO Load for Storage - that will dictate how you will carve up your SAN / underlying disks (ie do you use RAID1, RAID5 or RAID10).
In my experience for VM Load testing, the amount of load I generated one VM (with IOMeter) connected to our FC SAN, was so high, that comparing this to the IO Load generated by our production VM guests put together (that were all running on local storage) didn't come close to matching it! So I knew that our SAN was up to the task. And I am talking about Oracle Servers / SQL / CRM, etc ...  We are site that is double your size.
If you looking at SAN solutions. I highly recommend SanMelody (http://www.datacore.com/products/prod_SANmelody.asp) It is a great way of using disparate storage to build your SAN. In addition, to Windows, ESX, etc, we also have our Physical Novell Servers connected to our FC SAN via SanMelody, etc ... works great (but took a while to setup)..
0
 
LVL 32

Assisted Solution

by:nappy_d
nappy_d earned 50 total points
ID: 24857735
I have never been a fan of internal storage for any ESX environment except for a test lab.

VMWare recommends using the best practice of n+1 for ESX host planning. This practice means "n" is the number of ESX servers you want to have and then add 1 more for redundancy.

Shared storage is definitely the way to go.

I would point you in the direction of the Dell 2950III servers with dual quad core processors, and 32gb of RAM. Each processor core can support between 4-6 guest OS's.

For storage take a look at the Dell MD3000 DAS units. The MD3000 can support upto 4 servers connected for shared storage.

Along with this config, take a look at getting Virtual Center Foundations with 1year gold support.

For backups, consider adding VCB for accessing and backing up your ESX guests.
0
 
LVL 13

Expert Comment

by:bleeuwen
ID: 24857761
Keep in mind the MD3000 has SAS and/or SATA disks, this could be a performance bottleneck. You have also the i version (do not know that one) which can do iscsi. Both are supported bij the vwmare hcl list (www.vmware.com/go/hcl)
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24857800
If you've got internal drive bays and want to use them you can use LeftHand Virtual San Appliance to turn the internal sorage into an iSCSI SAN and use network RAID1 across two VMware servers running VSA to make the SAN redundant.

As far as hardware manufacturers go it doesn't really matter, they all perform similarly because they all use the same CPUs.

SFF disks rock, take a look at the HP MSA2324 in iSCSI, FC or SAS (up to 4 hosts max with SAS unless using blades) if you don't want a software iSCSI target running as a WM.
0
Scale it in WD Gold

With up to ten times the workload capacity of desktop drives, WD Gold hard drives employ advanced technology to deliver among the best in reliability, capacity, power efficiency and performance.

 
LVL 32

Expert Comment

by:nappy_d
ID: 24857974
Bleeuem, not sure I would consider 15K SAS drives slow by any stretch of the imagination.

I have in one environment an HP EVA 4400 san with SAS and it runs great.

The MD3000 comes in at about half the cost.
0
 
LVL 13

Expert Comment

by:bleeuwen
ID: 24858152
Sure it runs great, but every environment has it own needs. You need to design your environment to your needs, and ofcourse everything has it's price. If you build your san with multiple large 1 TB sas disks you have  less performance then multiple 300 GB sas disks. But for the same amount of storage you need much more disks, and more expansionsbay's (more money). So sometimes faster is better, but if you do not need it, it is a waste of money.
So in this case the MD3000 can be a great solution, but i do not know the requirements in this specific case.
0
 
LVL 32

Expert Comment

by:nappy_d
ID: 24858315
Keep in mind, the MD300, Like the HP MSA series or the HP EVA series are expandable.  So with all of these and other solutions, you can add additional trays for storage.  I would suggest you start with one try and fill it with drives.  As the environment grows, you can always add trays and configure additional LUNs as required.
0
 

Author Comment

by:jasonsfa98
ID: 24859570
It was like Christmas waking up to all this info, thanks guys.

One thing I do not quite understand with VMWare ESXi is it's own storage needs. For instance, do I even need physical disk installed in the server? It seems like it takes up so little space that any drives I get would be wasted. I suppose an SSD or something would be fine? Looks like HP does this.

Also, once I create a VM for say- Netware. Will it hurt performance to have the OS sitting on one datastore and the "data" volumes sitting on another? I ask because I always have the OS installed to a separate set of disk. It seems like there would be a lot of SAN traffic just to operate normally.
0
 
LVL 13

Expert Comment

by:bleeuwen
ID: 24859679
with esxi you also can run it on an usb key:
Download ESXi 4.o VMware-VMvisor-Installer-4.0.0-164009.x86_64.iso
Open image.tgz
Open image.tgz\usr\lib\vmware\installer\VMware-VMvisor-big-164009-x86_64.dd.bz2
Extract VMware-VMvisor-big-164009-x86_64.dd to your local hardrive
Attach the USB flash drive and make sure you no longer need the data on it
Use WinImage to transfer VMware-VMvisor-big-164009-x86_64.dd to the USB flash drive
       Disk->Restore Virtual Hard Disk image on physical drive&
       Select the USB flash drive (Warning: If you select the wrong disk you will lose data!)
       Select the image file VMware-VMvisor-big-164009-x86_64.dd
       Confirm the warning message
       Wait for the transfer to complete
Unplug the USB flash drive (Warning: If you forget to unplug the flash drive from the PC you might lose the data on your hard drives the next time you boot!)
Attach the USB flash drive to the machine you want to boot (Warning: If ESX Server 3i recognizes local drives, you might lose the data on it, so make sure you don´t need it anymore or unplug all hard drives!)
Turn the machine on and make sure the USB flash drive is selected as boot device  

 
0
 
LVL 32

Expert Comment

by:nappy_d
ID: 24859712
You can get servers with ESX embedded or you can just simply run it from a 2B USB key.

The best practices is you go iSCSI is to place this traffic on it's on vLAN or separate network switch from the rest of your network.

It will not hurt performance to have your data and OS on separate sets of disks.  The only performance hit to keep an eye on is the IO for read/writes to your shared storage.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Exchange server is not supported in any cloud-hosted platform (other than Azure with Azure Premium Storage).
Last article we focus in how to VMware: How to create and use VMs TAGs – Part 1 so before follow this article and perform the next tasks, you should read the first article how to create the TAG before using them in Veeam Backup Jobs.
Advanced tutorial on how to run the esxtop command to capture a batch file in csv format in order to export the file and use it for performance analysis. He demonstrates how to download the file using a vSphere web client (or vSphere client) and exp…
This Micro Tutorial steps you through the configuration steps to configure your ESXi host Management Network settings and test the management network, ensure the host is recognized by the DNS Server, configure a new password, and the troubleshooting…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now