ESXI: Expanding memory on HP DL380 G7 cluster and adding Cisco UCS nodes

Hello, I have two clusters, each with two DL 380 G7 servers on P67 Bios, running ESXi 5.1, and they now only have 48GB of RAM each (6x 8GB). The part number for my memory is: 500662-B21 (PC10600)

I was wondering which of the available 16GB modules available from HP provides the best value for money, in order to maximize the memory in the servers (12 x 16GB for 192GB per server). For example, the recent "647901-s21" is not listed as supported in the quick specs. Would it be a good idea to leave the existing 6x 8GB modules and add the only 6x 16GB, for a total of 176GB, or is this not recommended because all DIMM modules have to be the same size? 176GB would cover our needs at the moment.

Each server has Dual X5650 CPUs (2.66GHz/6-core/12MB/95W, DDR3-1333, HT, Turbo 2/2/2/2/3/3)

We are looking to expand these clusters, and I have a good offer for four Cisco UCS C220 M3 servers , each with dual E5-2620B 2.10 Ghz CPUs (6-core 80W), and 12x 16GB 1866Mhz RAM (PC3-14900, Product code UCS-MR-1X162RZ-A)

Can I add these in the same clusters as my current VMware environment, or do I need to put them on a new cluster because of the faster memory speed? my EVC compatibility mode is "Westmere" on both clusters.

I can either move the four new Cisco hosts in the second cluster, and then repurpose the existing HP hosts on the first cluster which already has HP hosts (option 2 below), or keep them separate like option 1 below:

Cluster 1 ------ Cluster 2
2x HP     ------- 2x HP

Future (option 1)    
Cluster 1 ------ Cluster 2
2x HP , 2x UCS     ------- 2x HP, 2x UCS

Future (option 2)
Cluster 1 ------ Cluster 2
4x HP     ------- 4x UCS

Thank you
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Can I add these in the same clusters as my current VMware environment, or do I need to put them on a new cluster because of the faster memory speed? my EVC compatibility mode is "Westmere" on both clusters.

You can add them all to the same cluster, and enable EVC mode, to ensure the same "processor baseline" is used for all servers. VMware designed EVC specially to mix and match processor versions, ensure they servers user wither AMD or Intel, do not mix.
sk391Author Commented:
Thank you, and the fact that half the cluster will use 1866Mhz memory instead of 1333Mhz memory also wont' be an issue?
andyalderSaggar maker's framemakerCommented:
Maximising the RAM on the DL380 G7s would be 12 * 32GB quad rank = 384GB, you could also use 18 * 16GB dual rank for 288GB. I'd be tempted to keep the 8GB DIMMs that you already have and add 12*16GB dual rank for 240GB. You can use although it does throw out some weird results sometimes.

You do not have to keep all DIMMs the same size but you should have the same amount of RAM on each CPU and where possible the same amount of RAM on each channel. "Badly" populated unbalanced configurations aren't nearly as bad as Intel would have you believe, see "WHITE PAPER FUJITSU PRIMERGY SERVERS MEMORY PERFORMANCE OF XEON 5600 (WESTMERE-EP) BASED SYSTEMS" where they deliberately mis-configure their server to show that although the memory bandwidth STREAM benchmark shows poor performance the real world SPECint benchmark shows only minor performance loss with badly balanced RAM.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
The 7 Worst Nightmares of a Sysadmin

Fear not! To defend your business’ IT systems we’re going to shine a light on the seven most sinister terrors that haunt sysadmins. That way you can be sure there’s nothing in your stack waiting to go bump in the night.

sk391Author Commented:
Thanks a lot andyalder, I checked the configurator and it's telling me that if I add 6x 16GB or 12x 16GB on top of my existing 6x 8GB, my memory speed will drop to 800Mhz from the current 1333Mhz.. would this be a concern? the workloads on the servers are mixed, it's a virtualized environment.

I think i'd rather remove the 48GB I have in order to add 12x 16GB and keep the memory speed @ 1067Mhz, right?

On the quickspecs of the server it only shows the Quad Rank 16GB as supported, but the configurator is showing me the Dual Rank that you mentioned..

12 x HP Option Part# 627812-B21 HP 16GB 2Rx4 PC3L-10600R-9 Kit (HP Module Part# 628974-081) LV
andyalderSaggar maker's framemakerCommented:
Memory speed doesn't matter all that much, you'll have to Google for the white paper I listed since I can't post URLs to PDFs but it shows that if 1066 MHz MHz is 100% speed then 800MHz is only 3% slower for SPECint in the "Relative Performance forDifferent Memory Speeds" chart.

As to the quickspecs the 16GB dual rank and 32GB quad rank are on the page below the rest, I missed them too first time I looked through it.
sk391Author Commented:
Got it, thank you! then it seems like the best option is keeping my 6x8 and adding 12 x 16GB dual rank as per your original suggestion.. at 800Mhz. This will give me 240GB per server, great!

One last thing please: my original ones are not "low voltage" as the Dual Rank 16GB (627812-B21), they are 500662-B21 (HP 8GB (1x8GB) Dual Rank x4 PC3-10600 (DDR3-1333) Registered CAS-9), can I still mix them with the new "LV" ones, or do *all* of them need to be "LV"?
andyalderSaggar maker's framemakerCommented:
Mixing LV (1.35v) and normal voltage (1.5v) is supported because LV work at both 1.35V and 1.5V, you can also mix different speeds and CAS timings since the CPU runs them all at the lowest common denominator.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.