Link to home
Start Free TrialLog in
Avatar of EGormly
EGormlyFlag for United States of America

asked on

Hardware for Vitualization - fairly large budget

I have been given a 15-20k budget for hardware to virtualize 6 servers with room for a few more in the future. 2 Webservers, 1 File server, 1 backup Server , 1 Mail Server and 1 SQL Server

I am not one for purchasing of the shelf items as they are extremely over priced.
So buying a Dell/HP is not in my plan.

So with that in mind I was looking at newegg for a barebones 2U 4CPU server.
I plan to add (4) 8 Core AMD Interlagos Processors, 256GB of Ram and 6 3TB 7200 SataIII drives, it will also have (2) 4 Port Nic cards for dedicated connections.  I also plan to duplicate the entire hardware setup for failover.

I would like to know expert opinions on this.
Is my choice of processor OK? Etc..

I am really good at building general computers and servers but this is my first foray into Virtualization, I have time t make it work, so once I get the hardware I can go through all my software options, for now I just want to avoid any mistakes or pitfalls I may not be aware of.
ASKER CERTIFIED SOLUTION
Avatar of HyperAdvisor
HyperAdvisor
Flag of Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
if your are not going to purchase tested and verified servers, I would avoid VMware ESXi, as you need a server from the HCL.

Server that supports 64 bit processors, Dual Processors with Intel-V or AMD-V, 6-8 Cores, and plenty of memory. Memory is often the bottleneck.

RAID controller.
Building a SERVER is extremely unwise.  How many HOURS of testing are you going to put in?  How are you going to ensure that if, 6 months down the line, the RAID controller starts acting funny, you can call the RAID controller manufacturer and a) get a replacement immediately and b) NOT have them blame the hard drive maker and c) not have the hard drive maker blame the RAID controller manufacturer all while your business is down because you felt Dell/HP was extremely over priced?

Buying from a major brand is insurance.  Insurance that you have ONE phone number to call and ONE parts supplier if something goes wrong.  DON'T TRUST YOUR BUSINESS TO DICE.  If you roll the right number, everything works.  If you miss, you bring down the entire company potentially for days.

As for being over-priced, sure, if you look online, Dell is over-priced.  CALL THEM.  Get a sales rep.  CONFIGURE online but then call and get a sales Rep and then have them quote the system. Do this in mid-april and you'll probably see 15-30% discount on the Online price.
For pure hypervisor virtualization, I would recomend to buy branded server (HP, DELL, IBM, Fujitsu). Otherwise, there's no guarantee it will work well.
If you insist on building your own server, generally be SURE to get SUPPORTED (by the hypervisor) SAS disk controller.
Avatar of EGormly

ASKER

While I appreciate the concern, I was not asking if I should buy branded vs. hand built.
With all due respect to all of you, you guys aren't helping answer the question I posted.
If you are not addressing the question, please do not post, as it seems to reduce the amount of possible pertinent answers.

To address the concerns brought up...

I am purchasing two duplicated sets of hardware for any "downtime" I might have that will be duplicated down to the screw, all the data will be on a NAS so there will be no downtime other than the time it takes to flip a switch.  Buying from Dell/HP surely guarantys me replacement of faulty hardware but it also takes a lot of time, I cannot afford to have all of my servers offline while Dell ships me a part overnight.

Back to the actul question though...

Is the AMD Interlagos 8 Core, 256GB ram and 6 3TB 7200 drives a good configuration for Virtualization?
Black and white on this issue I think

DON'T DO IT - YOU WILL REGRET IT
I'd look to Intel for the processors for ESXi, if you're going down the Hyper-V road then AMD is fine
Avatar of Member_2_231077
Member_2_231077

6 3TB 7200 drives would be good to backup onto, not much use in production at about 70 IOPS each. You need 10 or 15K enterprise drives.
I think your drives will be a little slow especially running SQL, you could cut back on the RAM 256Gb seems a little high, assuming you are going to dynamically allocate it to your VMs, this would save some money for 15K disks, see if you can get smaller capacity

You could also mix your RAID configuration, 7200 for low I/Os 15K for high I/Os and place the VMDK / VHD files depending on requirements

With shared storage you could have 3 nodes in a HyperV cluster with under 100Gb in each, might be a better option than an offline failover

Questions at this stage

1 ESXi or Hyper-V
2 What OS are your VMs going to me
3 Are you going to use shared storage, if not what failover will you use
4 Do you plan on using any management tools if so which ones
Gets a bit expensive on Windows licenses having a 3 node cluster, would need 3 standard licenses for each VM or 3/4 of an enterprise license. Could use datacenter licenses but they'd need to reduce the number of CPUs or that would blow their whole budget before buying any hardware.
So Hyper-V on a single server - with some sort of Active/Passive offline cluster fault tollerance

6 VMs
1 File server 16Gb ram - 1.5Tb disk
1 Email Server 16Gb ram 750Tb disk
2 Web Servers 4Gb RAM 200Gb Disk x 2
1 Backup Server 16Gb RAM 1Tb Disk
1 SQL Server 16Gb ram 750Gb Disk

Maybe a few more in the future, all Windows Server 2K8R2

Thats a total VM RAM requirement of 72Gb plus the host of another 16Gb = 88Gb

I'd have 2 RAIDs one for email and SQL running 15K drives, the other running 10K or 7.2K for the rest

You could also put one of teh file server drives on the 15K RAID so you could have a faster share if you have the space

I have over 200 VMs at the moment. RAM restricts you from adding more VMs but disk speed is always the bottleneck when the VMs are running

You can always add new more RAM when you want to add more VMs, hard drives aren't as easy to replace
I don't see much benefit to tiered storage having different disks and speeds for different jobs; I would put the whole lot on one set of spindles (split into several logical disks) so that should Exchange for example not be doing much then SQL has more IOPS available to it and vice versa. Requires a good RAID controller, I probably get a top end one from LSI so you can use online capacity expansion if needed to add more disks for both improved speed and capacity.
Just a suggestion, if your budget restricts you on disks it's a way to get the most out of it

If you can go for 15K for all drives then I'd do it, for sure
With all due respect, you are asking a question on s public forum where others can read.the answers. FAILING to point out how unwise it is to build your own server and hos much more expensive it is to buy two of everything rather than a name brand with 4 hour response time on support would be doing others who later read this question looking for guidance.

You fail to specify user count or workload beyond the basics of server job. I think anyone making vm hardware OS config recommendations is guessing.  Will 10 people be using the SQL server or 200.  What about the file server? Are your web servers getting 10 hits per day or 10 million.  And with a budget like this why on earth aren't you doing a vm cluster?

You need to analyze the loads on your existing servers before you decide what kind of vm solution you'll implement - we are not with you and don't gave access to the performance data from your systems to tell you if what you're trying to do is adequate.

It really sounds to me like your experience and knowledge in this area for a network that appears to be well into most definitions of medium size is currently inadequate.  I strongly advise you contact a consulting firm who can analyze your network in person and advise you what you need and should do.
First things first HP & Dell Servers are expensive for a reason they use expensive components.

if you think you can use half a dozen Eco Green SATA III HDD and the motherboards SATA controller you are sadly mistaken.

The main problem with HDD's is the seek time with 6+ virtual servers running on the same machine this could be a real problem.

You are going to need a Serious SAS HDD Controller with enterprise class SATA II HDD's I don't know of any that are 3TB SATA III there are some 2TB SATA II drives. I'm guessing the machines are going to be on 24/7 Standard SATA drives aren't designed for here are some Newegg product codes to look at.
N82E16822149217, N82E16822148610, N82E16822136579

As for those saying you need 15K drives these people are out of date you really need to look at enterprise class SSD's like the OCZ Z-Drive R4(N82E16820227772) this will make a huge difference and allow you to use cheaper drives for the mass storage.

Even if you cant stretch to enterprise class SSD's then there are cheaper alternatives there seek times beat the fastest 15K HDD's.

Good luck you probably wont save any money but will get a better system in the long run.


P.S.
Just had a quick look on NewEgg I don't think you are being vary realistic with your budget

SUPERMICRO AS-2042G-6RF 2U Rackmount Server Barebone Quad Socket G34 AMD SR5690/SR5670 DDR3 1600/1333/1066
Item #: N82E16816101321 $1,899.99

Seagate Constellation ES ST32000644NS 2TB 7200 RPM SATA 3.0Gb/s 3.5" Enterprise Internal Hard Drive -Bare Drive
Item #: N82E16822148610 $2,099.94 (6 @ $349.99 each)

Patriot Signature 16GB (2 x 8GB) 240-Pin DDR3 SDRAM DDR3 1333 (PC3 10600) ECC Registered Server Memory Model PS316G13ER2KH-E
Item #: N82E16820220554 $2,879.84 (16 @ $179.99 each)

AMD Opteron 6220 Interlagos 3.0GHz Socket G34 115W 8-Core Server Processor OS6220WKT8GGUWOF
Item #: N82E16819113028 $2,239.96 (4 @ $559.99 each)

areca ARC-1223-8I PCI-Express 2.0 x8 Low Profile SATA / SAS 8-Port PCIe 2.0 Internal SAS/SATA RAID Controllers
Item #: N82E16816151104      $470.00

Subtotal:       $9,589.73

& this doesn't include a SSD

P.P.S.
Don't forget to budget for the UPS.
We are not out of date, that much capacity using enterprise SSDs would far exceed the budget. SSDs are out of date anyway, for performance you can use NAND flash on a PCIe card, cost even more than SSDs though.
If your workloads are anywhere near mine, (and I have 650 users and run about 20 VMs per host), yo have too much RAM, too much CPU, and not enough disk IOPS. I have dual quad core Xeon 5400 servers with 72 GB RAM and 20 VMs and the physical CPU is at less than 10%. You can probably run everything on a single quad core processor. You can certainly do everything with an 8 core system. Anything more than that would be a waste of $$$ IMHO, unless you could justify more CPU. As has been pointed out, you have also proposed way too much RAM. Buy what you need now (plus a little more), and then add later as you need it and as prices come down. From what I can tell, 24 GB looks like enough RAM unless your web servers are running Java. You also have too many NICs. 4 NICs should be more than enough, especially since you probably only need 2 if you are not clustering.

Did you inlcude $$ for licensing Windows? Windows licenses can not be just transferred between hardware on a whim. My recommendation is to use single processor servers with a high core count, and then use Windows Datacenter to license each CPU on your hosts. Windows Datacenter is licensed per CPU, which is why you want to keep your CPU count down to just 1 per host. With that licensed CPU, you can run as many Windows Server VMs on that CPU as you want. The price for each Datacenter CPU license is about the same as a Windows Enterprise license. You should double check with Microsoft licensing to be sure that you can license just 1 CPU on a single CPU host.
As I understand it, you must get a MINIMUM of 2 CPU licenses with Datacenter.  Still, for a VM cluster, you'd only be able to use 2 machines with enterprise because of the transfer of license stuff.  So Datacenter would be best.
To run datacenter itself the minimum license is 2 CPUs/licenses, but for unlimited VM licenses you only need one CPU/license per server (if it's only got one CPU in it). There is a calculator at http://www.microsoft.com/windowsserver2003/howtobuy/licensing/calc_2.htm but I can't be bothered to install silverlight to use it.
So you're saying you can purchase two licenses and split them between 2 physical servers? I'd double check that understanding with MS
My Understanding is that if you INSTALL Datacenter, you need 2+ CPU licenses, but for VM licensing a single CPU host would only need a single CPU license.