New RAID/ZFS Recommendation for best Performance [ RAID60|ZFS]

Posted on 2013-07-01
Last Modified: 2016-12-08
I would like to get some feedback on the best (both performance and redundancy) solution based on the following hardware.

Scenario: This NAS server will be serving two XenServers running Windows 2012 VMs using iSCSI.

NAS Server: AMD 8120, 8 core 3.1 Ghz CPU with 16 GB RAM, Adaptec 6805 RAID card attached to 8 - 2 TB ReadyNAS drives, GigaByte 970A-D3 MB attached to 2 - 4 TB drives and 1 - 120 GB SSD (for OS) and a QUAD port Intel GB NIC (using link aggregation for iSCSI).

My original plan was to use the Adaptec 6805 as RAID60 (so I could loose 2 drives and still get a performance boost of striping ie. 60)

I have began reading a lot about ZFS; not I am totally confused as to what is the best route to go with FreeNAS.

Do I use the Adaptec controller and setup RAID 60, then install FreeNAS and will it then see the RAID60 as one drive ???


Do I disable the RAID of the Adaptec card, then install FreeNAS and have it use them as ZFS?

Really could use some educated guidance.....

For completeness, the two XenServer servers are identical hardware: GigaByte 970A-D3 MB, AMD 8150 8 core 4.0 Ghz CPU with 32 GB 1600 DDR3, 1 - 240 GB SSD S3 for OS, Intel DUO GB NIC (for iSCSI), DVDRW.
Question by:sdragoninc
  • 3
LVL 47

Accepted Solution

dlethe earned 500 total points
ID: 39291720
ZFS will provide better overall performance in most situations, and certainly higher data integrity.   It isn't fair comparing HW raid controller with ZFS.

The big reason is ZFS can use as much RAM as you have to prevent the need for some I/O.  It uses ram for read cache.  It can use a SSD for read or write cache. So if you want 100GB of read and/or write cache then you can do it.  Most cache you could get with the adaptec is a few MB.

But so much more to it...   You can tell ZFS to do hot snapshot backups, or keep an additional copy automatically of files in a certain directory.  Do that, and now you have more read load  balancing.  (Data in 2 places, typically different disks so when you read, it can give you the data for those files on whatever disk has least traffic at that instant)

You can do hot snapshot and online expansion in ZFS of just a filesystem.  You can't do that in adaptec hw raid.

The money you save going with a JBOD controller can be spent on more RAM and/or a small SSD for read cache or a ZIL (write intent log, or even a mirrored ZIL).

Bottom line, the Adaptec card will hurt performance if it is in any kind of RAID mode.  You must expose individual non-RAID disk drives.  If that card is not capable of presenting the Seagate, WD or whatever disks you have as n individual disk drives, then don't use the card. It will slow you down. The dumber the controller the better when using ZFS.
LVL 21

Expert Comment

ID: 39293075
In short, you should not use HW raid on top of SW raid, it should be one way or the other. You should use the SW raid provided by freenas for performance and scalability purpose.
If there is a drive failed, zfs have the ability to rebuild the array independently. If you need to move to different piece of hardware, you can export and import volumes.

I suggest you to take some time and read through the freenas hardware guide -
LVL 47

Expert Comment

ID: 39293109
ZFS is uniquely intolerant to even using HW raid in any way.   ZFS does a flush-on-write.  It does so for data integrity and to get around the RAID5 write hole.  (google that for an explanation).

So with the flush on write, this will kill hardware RAID performance, even if you just limit hardware RAID to RAID1.

Author Closing Comment

ID: 39333617
Ended up using ZFS2 and disabling the Adaptec RAID functions.  Had I known prior to purchasing, I would have bought a simpler SATA expansion board instead of RAID. Lesson learned.
LVL 47

Expert Comment

ID: 39333630
You might want to look deeper.   I do not know off top of my head if the Adaptec controller ends up virtualizing single disks.  It may do so.

Do the drives appear to ZFS as whatever make/model they are supposed to be, with the full number of blocks as documented in the specs?  (You can run the solaris format command to query such information.   It won't format the disks unless you specifically tell it to,  so select the drive and look at the # of blocks & make/model)

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
centos linux 65 159
linux SFTP 8 63
How does PHP Storm display on Linux high resolution laptops? 1 39
Questions about the Dell PERC h700 RAID card & RAID cards in general. 6 40
INTRODUCTION The purpose of this document is to demonstrate the Installation and configuration, of the HP EVA 4400 SAN Storage. The name , IP and the WWN ID’s used here are not the real ones. ABOUT THE STORAGE For most of you reading this, you …
Data center, now-a-days, is referred as the home of all the advanced technologies. In-fact, most of the businesses are now establishing their entire organizational structure around the IT capabilities.
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

830 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question