New RAID/ZFS Recommendation for best Performance [ RAID60|ZFS]

Posted on 2013-07-01
Last Modified: 2016-12-08
I would like to get some feedback on the best (both performance and redundancy) solution based on the following hardware.

Scenario: This NAS server will be serving two XenServers running Windows 2012 VMs using iSCSI.

NAS Server: AMD 8120, 8 core 3.1 Ghz CPU with 16 GB RAM, Adaptec 6805 RAID card attached to 8 - 2 TB ReadyNAS drives, GigaByte 970A-D3 MB attached to 2 - 4 TB drives and 1 - 120 GB SSD (for OS) and a QUAD port Intel GB NIC (using link aggregation for iSCSI).

My original plan was to use the Adaptec 6805 as RAID60 (so I could loose 2 drives and still get a performance boost of striping ie. 60)

I have began reading a lot about ZFS; not I am totally confused as to what is the best route to go with FreeNAS.

Do I use the Adaptec controller and setup RAID 60, then install FreeNAS and will it then see the RAID60 as one drive ???


Do I disable the RAID of the Adaptec card, then install FreeNAS and have it use them as ZFS?

Really could use some educated guidance.....

For completeness, the two XenServer servers are identical hardware: GigaByte 970A-D3 MB, AMD 8150 8 core 4.0 Ghz CPU with 32 GB 1600 DDR3, 1 - 240 GB SSD S3 for OS, Intel DUO GB NIC (for iSCSI), DVDRW.
Question by:sdragoninc
  • 3
LVL 47

Accepted Solution

dlethe earned 500 total points
ID: 39291720
ZFS will provide better overall performance in most situations, and certainly higher data integrity.   It isn't fair comparing HW raid controller with ZFS.

The big reason is ZFS can use as much RAM as you have to prevent the need for some I/O.  It uses ram for read cache.  It can use a SSD for read or write cache. So if you want 100GB of read and/or write cache then you can do it.  Most cache you could get with the adaptec is a few MB.

But so much more to it...   You can tell ZFS to do hot snapshot backups, or keep an additional copy automatically of files in a certain directory.  Do that, and now you have more read load  balancing.  (Data in 2 places, typically different disks so when you read, it can give you the data for those files on whatever disk has least traffic at that instant)

You can do hot snapshot and online expansion in ZFS of just a filesystem.  You can't do that in adaptec hw raid.

The money you save going with a JBOD controller can be spent on more RAM and/or a small SSD for read cache or a ZIL (write intent log, or even a mirrored ZIL).

Bottom line, the Adaptec card will hurt performance if it is in any kind of RAID mode.  You must expose individual non-RAID disk drives.  If that card is not capable of presenting the Seagate, WD or whatever disks you have as n individual disk drives, then don't use the card. It will slow you down. The dumber the controller the better when using ZFS.
LVL 21

Expert Comment

ID: 39293075
In short, you should not use HW raid on top of SW raid, it should be one way or the other. You should use the SW raid provided by freenas for performance and scalability purpose.
If there is a drive failed, zfs have the ability to rebuild the array independently. If you need to move to different piece of hardware, you can export and import volumes.

I suggest you to take some time and read through the freenas hardware guide -
LVL 47

Expert Comment

ID: 39293109
ZFS is uniquely intolerant to even using HW raid in any way.   ZFS does a flush-on-write.  It does so for data integrity and to get around the RAID5 write hole.  (google that for an explanation).

So with the flush on write, this will kill hardware RAID performance, even if you just limit hardware RAID to RAID1.

Author Closing Comment

ID: 39333617
Ended up using ZFS2 and disabling the Adaptec RAID functions.  Had I known prior to purchasing, I would have bought a simpler SATA expansion board instead of RAID. Lesson learned.
LVL 47

Expert Comment

ID: 39333630
You might want to look deeper.   I do not know off top of my head if the Adaptec controller ends up virtualizing single disks.  It may do so.

Do the drives appear to ZFS as whatever make/model they are supposed to be, with the full number of blocks as documented in the specs?  (You can run the solaris format command to query such information.   It won't format the disks unless you specifically tell it to,  so select the drive and look at the # of blocks & make/model)

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

this article is a guided solution for most of the common server issues in server hardware tasks we are facing in our routine job works. the topics in the following article covered are, 1) dell hardware raidlevel (Perc) 2) adding HDD 3) how t…
Join Greg Farro and Ethan Banks from Packet Pushers ( and Greg Ross from Paessler ( for a discussion about smart network …
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…

910 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now