Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win


Seeking storage solution advice for VMWare ESXi rollout.

Posted on 2009-07-14
Medium Priority
Last Modified: 2012-05-07
The time has come to replace our servers and I am looking to take advantage of virtualization. I am having a hard time deciding whether or not I want to go with servers that have plenty of (yet finite) internal storage or go with small (1U) servers and a SAN. I've been toying with VMWare ESXi and so far, I love it. I noticed that it will allow me to add networked storage as datastores to store virtual machines.

Since I am looking at about 3 TB of total storage I was thinking that a SAN and many smaller disk may be the way to go for performance. However, I can get 3 TB in a single 2U server with SFF disk. Below is a list of all the servers I need to virtualize...

4 - Novell Netware 6.5 (to be upgraded to Suse Linux ES 10 SP2 with OES)
6 - Windows Server 2003 (already running under MS Virt Server 2005)
1 - Suse Linux ES 10 SP2

Combined current storage usage is about 1.5 TB so I want to double that across the board to 3 TB total. In order to plan for a spike in data storage I am leaning towards providing about 4TB. I serve about 100 users with mainly File/Print/E-Mail. We do have a few specialized applications that require the Windows boxes.

Does anyone have advice on whether to go with a couple large form factor machines with internal storage or a SAN? If you like SANs do you recommend iSCSI or Fibre?
Question by:jasonsfa98
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 24

Accepted Solution

ryder0707 earned 600 total points
ID: 24855697
For SAN the backup is easier to manage especially if you decide to use VCB with your favourite backup software
Another advantage of have SAN as storage is you can share your storage among multiple hosts and use the esx advance feature, this is where the fun begins!
Both iscsi and fibre is good technology, everyone thinks that fibre is really fast(well it is)
iscsi can be fast too, depending on your design and budget, if you can allocate a pure dedicated IP network for your iscsi storage, it will be fast and cheap as well
and you are right, smaller disks for your VM are good for performance, in several enterprise customers i supported before, they normally present the LUN as required/needed, not really an estimation, so if you think you want 300GB now for 3 VMS
you can add a new 500GB datastore later for another few VMs
i always like to work with storage engineer, we may think that we know a lot about storage but when we deal with a dedicated storage guy, we'll learn a lot more :)

LVL 13

Assisted Solution

bleeuwen earned 600 total points
ID: 24856855
I would go for ESX servers with shared storage (ryder0707 says, iscsi or FC does not matter, it depends on your needs and budget). The advantage with a san is that you can expand. You can now buy what you need and next year, if you need more, expand it.
If you need more processing power, add an ESX server to it. If more storage is needed, add disks or an expansion bay. Offcourse this will cost you money at first, but gives you a lot of flexibility later on. Advantages of more ESX servers with a san is that you can use vmotion. When the server needs service, you can vmotion your vm's off to an other esx and bring the esx down or do what ever you like.
11 virtuals normally can run on one hardware box (depending on your load offcourse), but then it's normally full and you do not have room for expansion. Begin with two ESX servers and a shared storage, expand later. If you need to watch your money, you can go for a single esx with local storage, but a few advantages are gone. When you have a hardware failure, all virtuals are down. If you want to expand and need a second ESX box, and you want to vmotion, you still need shared storage and rebuild your environment.
Watch performance though. IF you run 11 virtuals on the same disks (local storage), it needs IO. It could be that your local disks aren't that fast enough to provide you with enough io to give you the performance you need (disk bottleneck).
LVL 21

Assisted Solution

za_mkh earned 600 total points
ID: 24857201
In addition to the above, be aware that the max size for a LUN or datastore that ESX/i supports is 2TB. You can use Extents to increase this to a total of 64TB but think about that too when you do this.
We currently have a few Novell Servers running on ESX and they are great. At the moment, they the IPP/Zenworks servers and soon our Fileservers too.
Check your IO Load for Storage - that will dictate how you will carve up your SAN / underlying disks (ie do you use RAID1, RAID5 or RAID10).
In my experience for VM Load testing, the amount of load I generated one VM (with IOMeter) connected to our FC SAN, was so high, that comparing this to the IO Load generated by our production VM guests put together (that were all running on local storage) didn't come close to matching it! So I knew that our SAN was up to the task. And I am talking about Oracle Servers / SQL / CRM, etc ...  We are site that is double your size.
If you looking at SAN solutions. I highly recommend SanMelody (http://www.datacore.com/products/prod_SANmelody.asp) It is a great way of using disparate storage to build your SAN. In addition, to Windows, ESX, etc, we also have our Physical Novell Servers connected to our FC SAN via SanMelody, etc ... works great (but took a while to setup)..
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

LVL 32

Assisted Solution

nappy_d earned 200 total points
ID: 24857735
I have never been a fan of internal storage for any ESX environment except for a test lab.

VMWare recommends using the best practice of n+1 for ESX host planning. This practice means "n" is the number of ESX servers you want to have and then add 1 more for redundancy.

Shared storage is definitely the way to go.

I would point you in the direction of the Dell 2950III servers with dual quad core processors, and 32gb of RAM. Each processor core can support between 4-6 guest OS's.

For storage take a look at the Dell MD3000 DAS units. The MD3000 can support upto 4 servers connected for shared storage.

Along with this config, take a look at getting Virtual Center Foundations with 1year gold support.

For backups, consider adding VCB for accessing and backing up your ESX guests.
LVL 13

Expert Comment

ID: 24857761
Keep in mind the MD3000 has SAS and/or SATA disks, this could be a performance bottleneck. You have also the i version (do not know that one) which can do iscsi. Both are supported bij the vwmare hcl list (www.vmware.com/go/hcl)
LVL 56

Expert Comment

ID: 24857800
If you've got internal drive bays and want to use them you can use LeftHand Virtual San Appliance to turn the internal sorage into an iSCSI SAN and use network RAID1 across two VMware servers running VSA to make the SAN redundant.

As far as hardware manufacturers go it doesn't really matter, they all perform similarly because they all use the same CPUs.

SFF disks rock, take a look at the HP MSA2324 in iSCSI, FC or SAS (up to 4 hosts max with SAS unless using blades) if you don't want a software iSCSI target running as a WM.
LVL 32

Expert Comment

ID: 24857974
Bleeuem, not sure I would consider 15K SAS drives slow by any stretch of the imagination.

I have in one environment an HP EVA 4400 san with SAS and it runs great.

The MD3000 comes in at about half the cost.
LVL 13

Expert Comment

ID: 24858152
Sure it runs great, but every environment has it own needs. You need to design your environment to your needs, and ofcourse everything has it's price. If you build your san with multiple large 1 TB sas disks you have  less performance then multiple 300 GB sas disks. But for the same amount of storage you need much more disks, and more expansionsbay's (more money). So sometimes faster is better, but if you do not need it, it is a waste of money.
So in this case the MD3000 can be a great solution, but i do not know the requirements in this specific case.
LVL 32

Expert Comment

ID: 24858315
Keep in mind, the MD300, Like the HP MSA series or the HP EVA series are expandable.  So with all of these and other solutions, you can add additional trays for storage.  I would suggest you start with one try and fill it with drives.  As the environment grows, you can always add trays and configure additional LUNs as required.

Author Comment

ID: 24859570
It was like Christmas waking up to all this info, thanks guys.

One thing I do not quite understand with VMWare ESXi is it's own storage needs. For instance, do I even need physical disk installed in the server? It seems like it takes up so little space that any drives I get would be wasted. I suppose an SSD or something would be fine? Looks like HP does this.

Also, once I create a VM for say- Netware. Will it hurt performance to have the OS sitting on one datastore and the "data" volumes sitting on another? I ask because I always have the OS installed to a separate set of disk. It seems like there would be a lot of SAN traffic just to operate normally.
LVL 13

Expert Comment

ID: 24859679
with esxi you also can run it on an usb key:
Download ESXi 4.o VMware-VMvisor-Installer-4.0.0-164009.x86_64.iso
Open image.tgz
Open image.tgz\usr\lib\vmware\installer\VMware-VMvisor-big-164009-x86_64.dd.bz2
Extract VMware-VMvisor-big-164009-x86_64.dd to your local hardrive
Attach the USB flash drive and make sure you no longer need the data on it
Use WinImage to transfer VMware-VMvisor-big-164009-x86_64.dd to the USB flash drive
       Disk->Restore Virtual Hard Disk image on physical drive&
       Select the USB flash drive (Warning: If you select the wrong disk you will lose data!)
       Select the image file VMware-VMvisor-big-164009-x86_64.dd
       Confirm the warning message
       Wait for the transfer to complete
Unplug the USB flash drive (Warning: If you forget to unplug the flash drive from the PC you might lose the data on your hard drives the next time you boot!)
Attach the USB flash drive to the machine you want to boot (Warning: If ESX Server 3i recognizes local drives, you might lose the data on it, so make sure you don´t need it anymore or unplug all hard drives!)
Turn the machine on and make sure the USB flash drive is selected as boot device  

LVL 32

Expert Comment

ID: 24859712
You can get servers with ESX embedded or you can just simply run it from a 2B USB key.

The best practices is you go iSCSI is to place this traffic on it's on vLAN or separate network switch from the rest of your network.

It will not hurt performance to have your data and OS on separate sets of disks.  The only performance hit to keep an eye on is the IO for read/writes to your shared storage.

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When we purchase storage, we typically are advertised storage of 500GB, 1TB, 2TB and so on. However, when you actually install it into your computer, your 500GB HDD will actually show up as 465GB. Why? It has to do with the way people and computers…
Many businesses neglect disaster recovery and treat it as an after-thought. I can tell you first hand that data will be lost, hard drives die, servers will be hacked, and careless (or malicious) employees can ruin your data.
Teach the user how to use create log bundles for vCenter Server or ESXi hosts Open vSphere Web Client: Generate vCenter Server and ESXi host log bundle:  Open vCenter Server Appliance Web Management interface and generate log bundle: Open vCenter Se…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial

618 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question