Solved

Vmware ESXi - hardware selection to backup

Posted on 2008-11-03
9
3,982 Views
Last Modified: 2010-04-21
Hi all
          I am going to throw a consulting question which I think will be useful for me and also for anyone looking to implement a virtual solution like us (small/medium business solution). I hope your answers will be a guide from start to end for anyone looking to virtualise and I am sure we can build our base on those.
      We are looking to virtualise 3 servers (Web, DB and file server) initially with the future potential of adding 3 more servers (including Exchange and AD). The existing  loads on these servers are as follows
Web server  Data size :10 GB, Processor usage: 20% normal and peak at 70% (peaking for 30 minutes daily)
DB server -    Data size :15 GB, Processor usage: 50% normal and peak at 90% (Peaking for an average 3 hours daily)
File server -    Data size : 100 GB, Processor usage: 60% normal and peak at 80% Peaking for an average 3 hours daily)

     The budget we have currently available to buy a server, virtualise (ESX3) or ESXi, SAN, backup or DR solution is £6000. (Plus an addition of £1000 in case..)

Please find the list of questions below. I just split the questions to make it clear. Please let me know the approximate pricing for each component as well.
1.      Do you think the above budget will be enough for this project?
2.      What spec of servers we will be looking for based on the above requirements?
3.      Is it wise to put all the above servers into one physical server or go for 2 servers?
4.      What type of SAN you will recommend (max size we are looking for is 2TB)?
5.      What connectivity on the SAN (Iscsi or Fibre)?
6.      What gigabit switch to use to connect the SAN to the host?
7.      Can we use our existing layer3 switch?
8.      What platform to use (ESX3 or ESXi)
9.      What backups and recovery method to choose? (we would like to have both Vm machine  level and file level backup and restore, I dont know which hypervisor (ESXi or ESX3) fits for this purpose?
10.      What kind of monitoring tools we use for the Vms?


Regards
Matheen




       

   
0
Comment
Question by:ExchangeGroup
  • 5
  • 4
9 Comments
 
LVL 8

Accepted Solution

by:
aldanch earned 500 total points
Comment Utility
1.      Do you think the above budget will be enough for this project?

We will need to be creative with this budget. Most of it will be spent on hardware / sofware (HW/SW) licensing costs.

2.      What spec of servers we will be looking for based on the above requirements?

Minimum Starter
2U-4U Rackmounted server (Based on Dell PowerEdge 2950)
CPU = One 2-3GHz Quad core
RAM = 8GB RAM at 667 MHz
NIC = 2-port Gb NICs (onboard)
HDD = 6 x 146GB 10k RPM SAS drives
HBA = 2-port iSCSI HBA (Qlogic) with Jumbo Frame support

Recommended Starter
2U-4U Rackmounted server (Based on Dell PowerEdge 2950)
CPU = 2-way or 4-way 2-3GHz Intel Quad core
RAM = 16GB 667MHz or higher
NIC = 4-6 Gb NICs
HDD = 6x400GB 10k RPM or 6x450GB 15k RPM SAS in a RAID5 + Hotspare (Try to stick with 10k or 15k RPM SAS drives)*
HBA = 2 1-port iSCSI HBA with Jumbo Frame Support

*HDD's not needed if you have a SAN in place. You can buy servers with ESX embedded (must specify when working with vendor of choice - Dell, HP, etc) which will allow you to run a completely diskless server. This will redirect storage costs (if needed) to the SAN and drive your server costs down.


3.      Is it wise to put all the above servers into one physical server or go for 2 servers?

It's better if you have two (single point of failure with one server). Realistically, you can start with a smaller configuration and scale up (upgrade existing HW - i.e. CPU, Memory, NIC, HDD) before you scale out (buy more HW/SW - i.e. another Server, an iSCSI or Fibre Channel SAN, Gigabit, 10-Gigabit, or FC Switch, iSCSI or FC HBAs, another UPS, etc)

One server will serve your needs based on budget costs. Plan to bring in a 2nd server by 2009 Q2, Q3, or Q4 to keep initial costs within target budget and to be on par with additional VMs to provision. Ensure to have enough Windows licenses (1 Windows Enterprise = 4 virtual machines per socket OR 1 Windows Datacenter = unlimited virtual machines per socket). Minimum starter requires at least 1 Enterprise license (2 if bringing in remaining VMs), whereas Recommended starter requires double the licenses.

If you choose the minimum starter and performance becomes an issue, scale up - 2nd quadcore CPU, More memory, more NICs - before scaling out.

If you want to acquire two servers, then do two Minimum starters to get started. Make necessary hardware reductions (slower CPU, less memory) in order to get the most out of both servers.

4.      What type of SAN you will recommend (max size we are looking for is 2TB)?

NetApp's StoreVault
http://www.netapp.com/us/products/storage-systems/s-family/

LeftHand Network's Virtual SAN - build your own server + storage, install ESXi, run this VM
http://www.lefthandnetworks.com/home.aspx
http://www.vmware.com/appliances/directory/1022

FalconStor's NSS Virtual SAN - build your own server + storage, install ESXi, run this VM
http://www.falconstor.com/en/index.cfm?CFID=4303732&CFTOKEN=86862201#
http://www.vmware.com/appliances/directory/15140

Datacore's SANMelody
http://www.datacore.com/products/prod_SANmelody.asp

5.      What connectivity on the SAN (Iscsi or Fibre)?

iSCSI (Future connectivity via 10-Gigabit when prices are more affordable)
Use iSCSI HBA (Qlogic) with redundant paths (MPIO)
Fully redundant = 2 controllers (all SAN components have redundant PSUs connected to different UPSes), 2 switches (either dedicated or a VLAN config to isolate iSCSI traffic), 2 HBAs installed on Server

6.      What gigabit switch to use to connect the SAN to the host?

For redundancy, use two dedicated switches or switches that support VLANs in order to isolate the iSCSI traffic. Recommend that all devices support Jumbo Frames. Two switches will help eliminate SOF (single point of failure) on the switch side. Fully redundant configuration requires multiple devices and multiple paths. If redundancy not an issue, then single Gigabit switch (For example, Cisco Catalyst 3560 or HP Procurve or Juniper )

7.      Can we use our existing layer3 switch?

Yes. Set up VLAN for iSCSI traffic (SAN to Server). Ensure you have enough ports for both iSCSI and network connections.

8.      What platform to use (ESX3 or ESXi)?

Either one, although ESX3x has better support for image/file level backups with 3rd party solutions. However, ESXi is replacing ESX3x as the platform of choice and has a smaller footprint (32MB) than it's Linux alternative (2GB). Takes about 15 minutes to configure. Can purchase server with ESX embedded. Execute commands via RemoteCLI. Start with ESXi 3.5 (free version) for 1st server. Purchase foundation, standard or enterprise license when bringing in a second server along with VirtualCenter (about $6000).

9.      What backups and recovery method to choose? We would like to have both VM machine level and file level backup and restore, I don't know which hypervisor (ESXi or ESX3) fits for this purpose?

Free version - use RemoteCLI with scripts to automate schedule backups or VCB (requires at least Foundation license). Can use third party software with VCB to do image and file level backups. ESX3.5 is fully supported with most 3rd party vendors such as ESXpress (http://www.esxpress.com/) or VizionCore (http://www.vizioncore.com/) with ESXi in the process of getting support.

10.     What kind of monitoring tools we use for the VMs?

Your choice of SNMP (http://www.vmware.com/support/esx25/doc/sys_mgmt_links.html)

Dell OpenManage, HP OpenView, IBM Director, Fujitsu Siemens ServerView

You may also use the built-in monitoring tools in ESX through VI Client and RemoteCLI or Microsoft System Center Virtual Machine Manager (SCVMM) when it releases
0
 

Author Comment

by:ExchangeGroup
Comment Utility
Hi Aldanch

         Thanks for your detailed reply. I would like to get clarified on some of your points.

In Brief We decided to do like this way, let me know it can work and any downsides on this

In terms of servers we decided to go for 2 physical servers and have vmware HA. In terms of the SAN, I will buy 2 Direct attached storage and plug on to the 2 physical servers. Then using SAN melody or other softwrare you mentoned we can convert the storage into SAN and mirror the two DAS. For backup we will go for esxpress.

1.But for the server hardware, Is it ok if we go for 1 Quadcore 3.0 ghz processor for both the servers. If we do that then we need to buy only 2 processor license from vmware as opposed from 4 processor if we go for any Dual processors on the physical severs) Is it right? Will the processing power be ok to run 3 servers?

2. If we have 1 quadcore 3.0ghz processor that means we have 12Ghz processor power to assign for the vm's?

3. If we go for DAS, what minimum spec we need to have on the physical server's hard drive? Do we need to RAID the hard drive? Scsi or SAS hard drive?

regards
Exg
0
 
LVL 8

Expert Comment

by:aldanch
Comment Utility
1.But for the server hardware, Is it ok if we go for 1 Quadcore 3.0 ghz processor for both the servers. If we do that then we need to buy only 2 processor license from vmware as opposed from 4 processor if we go for any Dual processors on the physical severs) Is it right? Will the processing power be ok to run 3 servers?

Yes, that is fine. That'll eliminate the single point of failure on the server hardware and will enable you to fail over to one ESX server for planned or unplanned downtime. The downside is when it's time to scale up, it'll be double the hardware, but I'm sure you can build a case for the upgrade when the time comes.

Stick with the quad core processors. Intel just released their Core i7 processors (http://en.wikipedia.org/wiki/Intel_Core_3) and its line of processors is the first true quad core (Not 2 dual core processors on a single die like the Intel Core 2's) and is the successor to Intel Core 2. Make sure your ESX server's CPU are in the same family (I imagine that you'll spec them out identically, so this won't be a problem). Also, be sure to enable Intel's VT in the BIOS (sometimes they come shipped disabled, so just verify that it's enabled).

Did you decide to go ESXi or ESX 3.5 (Linux)? If ESXi, don't forget to have your server vendor include ESXi embedded in your specs

Regarding your VMs, make sure to have enough memory to support each application. Over provision your server hardware but don't overly zealous with memory in each VM (over provisioning memory/CPU on a VM actually hurts performance), just right size the VM's according to the application's best practices. That'll steer you to determining the right amount of RAM you'll need to install on your servers.

Keep in mind that since you're bringing in 2 servers with the intent of being able to fail over from one ESX server to another, you must have enough memory on each server to support ALL VMs deployed. So when you determine that 8 GB is all you need to run all your applications, then 10-16GB is what you're buying for EACH ESX server (Over provision the server, right-size the VMs). Otherwise, you'll have memory contention - VMs competing with each other for memory resources. Also, this will enable you to increase memory resources if need be. If all VMs can run on a single server (in your 2-server VI3 environment), then having a 2nd server will allow you to distribute your VMs across more resources.

2. If we have 1 quadcore 3.0ghz processor that means we have 12Ghz processor power to assign for the vm's?

Yes. You can set affinity rules so that each VM is assigned a particular CPU, but you shouldn't have to set this. Typically, your VMs should run fine with one vCPU, but like memory, spec out your VMs according to your application best practices. Start with one vCPU and add additional vCPUs when needed.

3. If we go for DAS, what minimum spec we need to have on the physical server's hard drive? Do we need to RAID the hard drive? Scsi or SAS hard drive?

The one I spec'd out is based on a Dell PowerEdge 2950. You will be setting up the RAID configuration. So for a 2U server like the 2950, that's either 6 3.5" SAS drives or 8 2.5" SAS drives. SAS is still SCSI; it just uses a faster bus (3 Gbps) and most servers ship with SAS, but you can opt for SCSI if price is an issue. Try and stick with 10k or 15k RPM drives for optimum performance. Factor spindle speed into your pricing.

Drive size will depend on your VM's requirements. You can choose which RAID configuration would work best for you. Use a RAID5 + Hotspare, RAID6 + Hotspare, RAID10 + Hotspare. I can't stress the Hotspare enough. Depending on which drive dimension you choose (3.5" or 2.5") and which speed (10k or 15k RPM) you decide will determine the drive size available. Check with your vendor for drive types available for your server configuration.

Note: You mention you're going with VMware HA. I take it this is the VI3 Standard license. This will work for your needs for now, but in the future, I highly suggest upgrading to Enterprise to take advantage of additional automation features (DRS, VMotion).

With a Standard license, you'll need to cold migrate (VMs are powered down then manually moved from one ESX server to another) your VMs. VMotion allows you to hot migrate them (move VM while it's running, requires central storage - SAN, NAS - to enable this and compatible CPU). Additionally, you would need to manually distribute your VMs according to the load they place on your ESX servers (DRS automates this for you).

Do plan to bring in VirtualCenter (about $6000) in the future because without it, you'll be managing each ESX server separately. Either run VirtualCenter on a separate server or as a VM within your VI3 environment. For now, managing each ESX server individually will work just fine (using the VI client).
0
 

Author Comment

by:ExchangeGroup
Comment Utility
Hi Aldanch

          I managed to increase the budget from £7000 to £10000. But still I have trouble fitting the costs with the budget. I had a look at Dell 2950 servers (with quadcore processor and 16gb RAM and 4 network cards) and it is easily going over £2500 for one server. I also had a look at  a hardware SAN and the basic is starting from £4000 etc. So what I am thinking now is to go for a quite decent spec server with more internal storage (then virtualise using software SAN). see the server spec, Dell price is £3400

--------------------------
PE 2950 III Quad Core Xeon X5450 (3.0GHz, 2x6MB, 1333MHz FSB
Riser with PCI Express Support (2x PCIe x8 slots; 1x PCIe x4 slot) 1 S
16GB (8x2GB Dual Rank DIMMs) 667MHz FBD 1 S
PE2950 III Additional Quad-Core Xeon X5450 (3.0GHz, 2x6MB, 1333MHz FSB) 1 S
No Floppy for x6 Backplane 1 S
300GB SAS 15k 3.5" HD Hot Plug 5 S
PE2950 III - Chassis 3.5HDD x6 Backplane 1 S
PERC 6/i, Integrated Controller Card x6 backplane 1 S
DVD-ROM Drive SATA with SATA Cable 1 S
PE2950 III Redundant Power Supply No Power Cord 1 S
Rack Power Distribution Unit Power Cord 1 S
Intel PRO 1000PT Dual Port Server Adapter, Gigabit NIC, Cu, PCIe x4 2 S
TCP/IP Offload Engine 2P 1 S
No Operating System 1 S
PE2950 Open Manage CD + Drivers 1 S
You have chosen not to take the Dell PowerEdge installation service 1 S
Rapid/Versa Rack Rails 1 S
PE2950 III C4 MSS R5 Add-in PERC 5/i / 6/i 1 S
PowerEdge Order - United Kingdom 1 S
Base Warranty 1 S
1Yr Basic Warranty - Next Business Day - Minimum Warranty
---------------------------------------------------------------------------

Do you think the spec is ok  and the price is justifable for the spec?

If we were to get 2 identical  servers, Can we virtualise the internal storage into a SAN by using software (san melody etc)  and mirror the internal  storage on the 2 servers to get

If we do that how redundant the servers are?  Can we use vmwae HA with the above seup?

Finally, What RAID connections we have to have on the physical server? Do we need to isolate the internal storage with the ESX installation drive?

Regards
EXG


0
Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

 
LVL 8

Expert Comment

by:aldanch
Comment Utility
Do you think the spec is ok  and the price is justifable for the spec?

The specs are great. I noticed that it had a 2nd Quadcore processor ($861 - check with Dell to confirm). While a 2nd processor will allow you to add more VMs, it may hamper your budget.

Bringing in a 2nd server will provide you with more benefits than just a CPU upgrade - more memory, more storage, more network connections. Budget-wise it'll be easier to bring in 2 single quadcore servers than 2 dual quadcore ones. Also, that would also mean buying additional Microsoft and VMware licenses. Since you're budget conscious, may be set up your servers with the following:

CPU = 1 Quadcore
RAM = 16 GB (or 8 GB)
HDD = 6x300GB 15k, 400GB 10k, 450GB 15k SAS or 8x146GB 10k SAS (2.5") (this is where it gets expensive)

Another thing I noticed is that I didn't see ESXi embedded. Make sure they include this unless you're planning on installing ESX 3.5 rather than ESXi.

If we were to get 2 identical  servers, Can we virtualise the internal storage into a SAN by using software (san melody etc)  and mirror the internal  storage on the 2 servers to get

You most certainly can, and SANMelody, LeftHand Networks, etc will allow you to do this

Here are some links regarding this:

Datacore
http://www.datacore.com/solutions/business_continuity.asp
http://www.datacore.com/VMstoragefoundation/default.asp
http://www.datacore.com/products/prod_SANmel_prerequisites.asp
http://communities.vmware.com/docs/DOC-1570
http://communities.vmware.com/message/1019406;jsessionid=03C317AF15F224305F8B66200059F265
http://www.las-solanas.com/storage_virtualization/vmware_esx_san_software_vm.php

LeftHand Networks
http://www.lefthandnetworks.com/vsa.aspx
http://engineering.xtravirt.com/index.php?option=com_content&task=view&id=79&Itemid=5 (reference architecture)

Here's another Virtual SAN Appliance vendor (VSA is free, pay for support)

Xtravirt
http://engineering.xtravirt.com/products/xtravirt-virtual-san.html
http://vmetc.com/2008/05/23/xtravirt-xvs-creates-a-free-san-out-of-local-esx-vmfs/
http://engineering.xtravirt.com/products.html

Make sure to follow best practices based on the Storage vendor (Datacore, LFN, Xtravirt, etc) you choose for setup

If we do that how redundant the servers are?  Can we use vmwae HA with the above seup?

If you're going for high availability, then 2 servers will provide you the redundancy you need. Since you're trying to bring in 2 servers, you can spec them out identically. However if you're factoring in hardware (servers, switches, cables, power, ups), software licensing (windows enterprise, vmware standard or enterprise, SAN software or VSA), and support costs (1-3 years hardware/software), it'll be difficult bring in 2 servers unless you sacrifice somewhere (hw, sw, support).

When you mention VMware HA, are you referring to the Acceleration Kit (VI Standard HA - http://www.vmware.com/solutions/smb/whats_new.html)? If so, with $10k there's no way to bring in this license along with the configuration for the hardware, licensing costs for Windows Server, and SAN storage software/appliance.

For now, you may want to consider just running ESXi on your servers (free version - make sure the ESXi embedded is part of your server spec) and planning an upgrade to an acceleration kit (VI Standard HA) later (for centralize management of both ESX servers and your VMs). This means that you'll be managing your VMs through a VI client logged into each ESX server. That way all your budget is focused on getting the right hardware configuration for your servers as well as the right licenses for your Windows Server (recommend Enterprise if you don't already have it) and SAN appliance.

Finally, What RAID connections we have to have on the physical server? Do we need to isolate the internal storage with the ESX installation drive?

This depends on how many drives you will have installed on the server.

If, say, you have 6 drives then setting up a RAID5 with hotspare is ideal (or RAID50 if drive space is not an issue)

If you have 8 drives (2.5") then setting up a RAID50 is probable (but no hotspare) but RAID5 with hotspare is fine also

As far as HDD size and speed, this is where you can be creative because the cost of filling up a PE2950 with 6 drives or 8 drives would bump the total cost per server to above $5000 (even without licensing for Windows or VMware).

I do recommend either 8 2.5" drives (10k RPM) or 6 3.5" drives (10k or 15k RPM), but your drive configuration will depend on your storage requirements (speed and total usable size)

For example,

6x300GB 15k = RAID5 + Hot Spare or RAID50 no HS = Approximately 1 TB of usable space = add $2700 to server cost
8x146GB 10k = RAID5 + Hot Spare or RAID50 no HS = Approximately 800 GB of usable storage = add $1348 to server cost

You don't have to isolate the internal storage.
0
 
LVL 8

Expert Comment

by:aldanch
Comment Utility
The important thing is that your virtual environment are spec'd out properly.

Server Config 1
2 Dell PE2950 III
1 CPU Quadcore
8 or 16 GB RAM
6x300GB 15k or 6x400GB 10k or 8x146GB 10k in a RAID5 with HS or RAID50 configuration
4-6 Gigabit NICs (2 onboard + 2-port or 4-port PCI-e card)
2x Power Supplies (separate cord for each connected to separate UPS)
Operating System: ESXi embedded or ESX 3.5 (Free version)
2 Windows Server 2003/2008 Enterprise licenses (1 for each installed CPU) to run 4 VM instances
SAN software = Xtravirt (Free), SANMelody ($$), LeftHand Networks ($$)

Server Config 2
1 Dell PE2950 III
2 CPU Quadcore
16 GB RAM
6x300GB 15k or 6x400GB 10k or 8x146GB 10k in a RAID5 with HS or RAID50 configuration
4-6 Gigabit NICs (2 onboard + 2-port or 4-port PCI-e card)
2x Power Supplies (separate cord for each connected to separate UPS)
Operating System: ESXi embedded or ESX 3.5 (Free version)
2 Windows Server 2003/2008 Enterprise licenses (1 for each socket) to run 4 VM instances
SAN software = Xtravirt (Free), SANMelody ($$), LeftHand Networks ($$)

Switch
VLAN for iSCSI and VMkernel (VMotion and Management) traffic on existing switch or dedicated switch (at least 4-ports or more), or cross over cable (ESX to ESX) for each traffic type
1 VLAN for iSCSI (2 NICs)
1 VLAN for VMkernel/VMotion/Management (2 NIC)
1 VLAN for physical network (2 NIC)
Consumed ports: 4-12 ports depending on server config (4-12 patch cables)
*For redundancy, there should be 2 of everything - servers, switches, UPS (2-4 power connections), etc - for multipathing support

Power
Each server power connection should be connected to a different UPS
0
 

Author Comment

by:ExchangeGroup
Comment Utility
Hi Aldanch

        Thank you very much for your detailed reply. I find you are very patient in explaining. I am sure I will take some of your suggestions and apply to our new virtuliasation project. I have also added another question 'Citrix xen5.0 or esx 3.5'. May be you can have a look and post your suggestions as well.

Regards
~EXg
0
 

Author Closing Comment

by:ExchangeGroup
Comment Utility
The support is excellent. The expert is patinet, gave detailed explanation and prompt as well.
0
 
LVL 8

Expert Comment

by:aldanch
Comment Utility
EXg,

You're welcome. Good luck with your virtualization project.

I'll take a look at your Xen-ESX question, too.
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

This is an issue that we can get adding / removing permissions in the vCSA 6.0. We can also have issues searching for users / groups in the AD (using your identify sources). This is how one of the ways to handle this issues and fix it.
HOW TO: Connect to the VMware vSphere Hypervisor 6.5 (ESXi 6.5) using the vSphere (HTML5 Web) Host Client 6.5, and perform a simple configuration task of adding a new VMFS 6 datastore.
Teach the user how to install ESXi 5.5 and configure the management network System Requirements: ESXi Installation:  Management Network Configuration: Management Network Testing:
Teach the user how to install vSphere Update Manager  Console to Windows system:  Install vSphere Update Manager: Configure vSphere Update Manager plug-in in vSphere Client: Verify vSphere Update Manager settings in vSphere Client:

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now