Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)
Course Of The Month
Become a Premium Member and unlock a new, free course in leading technologies each month.
Solved
ZFS: Nexentastor best practises for ESXi 5
Posted on 2012-12-28
Hi,
I just configured my HP Microserver 16GB NL 40 with Nexenta. There are 6 SSD's inside: 2 for OS (mirrored, 60 GB), 4 SSD's 120GB (Kingston, enterprise version), 3 SATA 250 GB 5400 RPM.
Before setting this up, I actually only was looking for a fast storage NAS in my lab. I ended up spending some weeks but actually I don't want to spend any more now. I want to focus again on Microsoft products which run on my vm's.
Therefore this question.
I saw loads of articles for best practises: using NFS3 instead of 4 for ESXI?, make cache/log with mirrored drives?, backup the storage using snapshots?, use ISCSI over NFS?
Now, as mentioned, I don't want to spend much more time on configuring, checking best performance, just have a best practice "jump start".
So please advise (configured ZRAID on 4 SSD's of 120GB, NFS; 3 SATA's in ZRAID too).
Note: I'll probably replace the SATA's with 3 or more barracuda 2TB 64 MB cache, green disks for having more storage.
J.
I just configured my HP Microserver 16GB NL 40 with Nexenta. There are 6 SSD's inside: 2 for OS (mirrored, 60 GB), 4 SSD's 120GB (Kingston, enterprise version), 3 SATA 250 GB 5400 RPM.
Before setting this up, I actually only was looking for a fast storage NAS in my lab. I ended up spending some weeks but actually I don't want to spend any more now. I want to focus again on Microsoft products which run on my vm's.
Therefore this question.
I saw loads of articles for best practises: using NFS3 instead of 4 for ESXI?, make cache/log with mirrored drives?, backup the storage using snapshots?, use ISCSI over NFS?
Now, as mentioned, I don't want to spend much more time on configuring, checking best performance, just have a best practice "jump start".
So please advise (configured ZRAID on 4 SSD's of 120GB, NFS; 3 SATA's in ZRAID too).
Note: I'll probably replace the SATA's with 3 or more barracuda 2TB 64 MB cache, green disks for having more storage.
J.
[X]
Welcome to Experts Exchange
Add your voice to the tech community where 5M+ people just like you are talking about what matters.
- Help others & share knowledge
- Earn cash & points
- Learn & ask questions
-
7
5
5
17 Comments
Active 1 day ago
Best practice is to get rid of ESXi entirely with Nexenta. You want to run native Nexenta on the physical machine.
As for using 'cuda green drives those disks are low duty cycle, and not designed for 24x7x365. If you DO use them, don't even think of doing anything other than RAIDZ2.
As for using 'cuda green drives those disks are low duty cycle, and not designed for 24x7x365. If you DO use them, don't even think of doing anything other than RAIDZ2.
Active today
you may find better performance using iSCSI LUNs, if your NFS performance is not as expected.
you will need to test performance in the VMs.
Benchmark software listed as before.
your current config looks okay. experiment with adding a ZIL log SSD to your SATA three, for increased performance, also SSD for cache
you will need to test performance in the VMs.
Benchmark software listed as before.
your current config looks okay. experiment with adding a ZIL log SSD to your SATA three, for increased performance, also SSD for cache
Active 1 day ago
You are wasting the SSDs on O/S disk.
Use one SSD for cache. Get two cheap, cheap desktop or even laptop SATA disks and use them mirrored for root pool. As for the rest, it really depends on what you are trying to do, but I would generally make it easy and go with single RAIDZ2 pool with 4 high capacity SATA disks, 3 SSDs, (mirrored pair for ZIL), 3rd for cache.
unmirrored ZIL is foolish and you risk major data loss if the ZIL disk fails. Don't do it. Losing the cache SSD does not risk data.
Buy less SSD and more RAM, as system memory is used for read cache and Nexenta/Solaris will use every bit of it.
Use one SSD for cache. Get two cheap, cheap desktop or even laptop SATA disks and use them mirrored for root pool. As for the rest, it really depends on what you are trying to do, but I would generally make it easy and go with single RAIDZ2 pool with 4 high capacity SATA disks, 3 SSDs, (mirrored pair for ZIL), 3rd for cache.
unmirrored ZIL is foolish and you risk major data loss if the ZIL disk fails. Don't do it. Losing the cache SSD does not risk data.
Buy less SSD and more RAM, as system memory is used for read cache and Nexenta/Solaris will use every bit of it.
Thanks for the input dlethe!
I do want to make it easy, general.
However, I do have the ssd s installed and don t like to start all over installing Nexenta afain on other disk (might do that some day but for now I want to play with my new born nas :-)).
I don t want more then 1 disk for redundancy (it s a lab after all, if 1disk down, I could shutdown whole nas and go to shop for other disk) UNLESS performance proves drastically.
That would mean raidz, not raidz2 I guess.
So you wouldn t make a raidz of ssd s then and use a mirror for zil and 1 disk for cache. Then I have 1 spare ssd 120 gb. Could use that in my laptop.
Note: microserver has already 16 gb ram which is max it can hold.
I do want to make it easy, general.
However, I do have the ssd s installed and don t like to start all over installing Nexenta afain on other disk (might do that some day but for now I want to play with my new born nas :-)).
I don t want more then 1 disk for redundancy (it s a lab after all, if 1disk down, I could shutdown whole nas and go to shop for other disk) UNLESS performance proves drastically.
That would mean raidz, not raidz2 I guess.
So you wouldn t make a raidz of ssd s then and use a mirror for zil and 1 disk for cache. Then I have 1 spare ssd 120 gb. Could use that in my laptop.
Note: microserver has already 16 gb ram which is max it can hold.
Active today
also note, that a single low power, low performing, AMD Neo N40L 1,5GHz dual core processor, is the bottleneck! also onboard nic does not support jumbo frames
my new Nexus phone performs faster than a ProLiant Microserver!
before you waste anymore time benchmark and test existing NAS versus Microserver, you may find your existing NAS has a faster processor.
my new Nexus phone performs faster than a ProLiant Microserver!
before you waste anymore time benchmark and test existing NAS versus Microserver, you may find your existing NAS has a faster processor.
Hanccocka, you always mentioned it s the fastest nas you ll find around? That s why I started with it in the first place ...
Active today
I have stated ZFS is the fastest NAS technology around, now being employed by other NAS vendors eg Thecus etc, even better when using SSD for ZIL and Cache.
without any published benchmarks with your current NAS, its difficult to compare, if you look back through my many posts you will see many references to the Microservers performance, you cannot expect the performance of the Microserver to be superior to a tower server because of its low spec.
if you also check forums, google, ProLiant Microservers performance bottleneck is limited by its design. again its a compromise of price, size and power versus performance.
and for its price, power, performance it packs a punch.
BUT, We do not have ANY tested performance metrics for your current setup, not that Ive seen.
We use these in OUR Research and Development, Training and Demo Labs for ALL VMware Products, performance is acceptable for us, we have no issues with ZFS, when using ESXi with NFS and iSCSI.
These SAN/ZFS based MicroServers replaced 1GB Fibre Channel SAN, using 1GB FC, and 10k and 15k rpm U320 SCSI disks. NFS and iSCSI performance was tested to be far superior to existing shared storage. Considering the cost of electricity, and performance, size of current storage in (old FC SAN), it was a no brainer to switch. (based on metrics ancd testing).
without any published benchmarks with your current NAS, its difficult to compare, if you look back through my many posts you will see many references to the Microservers performance, you cannot expect the performance of the Microserver to be superior to a tower server because of its low spec.
if you also check forums, google, ProLiant Microservers performance bottleneck is limited by its design. again its a compromise of price, size and power versus performance.
and for its price, power, performance it packs a punch.
BUT, We do not have ANY tested performance metrics for your current setup, not that Ive seen.
We use these in OUR Research and Development, Training and Demo Labs for ALL VMware Products, performance is acceptable for us, we have no issues with ZFS, when using ESXi with NFS and iSCSI.
These SAN/ZFS based MicroServers replaced 1GB Fibre Channel SAN, using 1GB FC, and 10k and 15k rpm U320 SCSI disks. NFS and iSCSI performance was tested to be far superior to existing shared storage. Considering the cost of electricity, and performance, size of current storage in (old FC SAN), it was a no brainer to switch. (based on metrics ancd testing).
Active 1 day ago
What is your I/O model? Be specific, ratio of reads vs writes vs sequential vs random I/O? Will you use deduplication or compression? Are these small data files?
If you are going all SSD, then just stripe 2 mirrors together. No need for ZIL or cache if you are all SSD, you need them if you go RAIDZ1 and those cheap slow SATA disks.
If you are going all SSD, then just stripe 2 mirrors together. No need for ZIL or cache if you are all SSD, you need them if you go RAIDZ1 and those cheap slow SATA disks.
Active 1 day ago
You can do both, it just depends on the I/O load. At some point CPU utilization may suffer, but if I/O utilization hits a ceiling first, then CPU load wont be a significant bottleneck.
I'll make a new ticket asap but nappit doesn't install ... just shows this:
admin@Nexenta:~$ wget -O - www.napp-it.org/nappit | perl
--14:33:49-- http://www.napp-it.org/nappit
=> `-'
Resolving www.napp-it.org... 80.67.28.3, 2a00:1158:0:300:9a63::1
Connecting to www.napp-it.org|80.67.28.3|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 23,126 (23K) [text/plain]
100%[=====================
14:33:49 (235.53 KB/s) - `-' saved [23126/23126]
Logging on with root gives "wrong command".
admin@Nexenta:~$ wget -O - www.napp-it.org/nappit | perl
--14:33:49-- http://www.napp-it.org/nappit
=> `-'
Resolving www.napp-it.org... 80.67.28.3, 2a00:1158:0:300:9a63::1
Connecting to www.napp-it.org|80.67.28.3|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 23,126 (23K) [text/plain]
100%[=====================
14:33:49 (235.53 KB/s) - `-' saved [23126/23126]
Logging on with root gives "wrong command".
Featured Post
We value your feedback.
Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!
Question has a verified solution.
If you are experiencing a similar issue, please ask a related question
In this step by step tutorial with screenshots, we will show you HOW TO: Enable SSH Remote Access on a VMware vSphere Hypervisor 6.5 (ESXi 6.5). This is important if you need to enable SSH remote access for additional troubleshooting of the ESXi hos…
Ransomware is a malware that is again in the list of security concerns. Not only for companies, but also for Government security and even at personal use. IT departments should be aware and have the right knowledge to how to fight it.
Teach the user how to delpoy the vCenter Server Appliance and how to configure its network settings
Deploy OVF: Open VM console and configure networking:
Teach the user how to configure vSphere clusters to support the VMware FT feature
Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
Suggested Courses
Earn Certification
HTML5 Specialist - Certification
Free withPremium
Course of the Month6 days, 13 hours left to enroll
622 members asked questions and received personalized solutions in the past 7 days.
Join the community of 500,000 technology professionals and ask your questions.