New RAID/ZFS Recommendation for best Performance [ RAID60|ZFS]

Posted on 2013-07-01
Last Modified: 2013-07-17
I would like to get some feedback on the best (both performance and redundancy) solution based on the following hardware.

Scenario: This NAS server will be serving two XenServers running Windows 2012 VMs using iSCSI.

NAS Server: AMD 8120, 8 core 3.1 Ghz CPU with 16 GB RAM, Adaptec 6805 RAID card attached to 8 - 2 TB ReadyNAS drives, GigaByte 970A-D3 MB attached to 2 - 4 TB drives and 1 - 120 GB SSD (for OS) and a QUAD port Intel GB NIC (using link aggregation for iSCSI).

My original plan was to use the Adaptec 6805 as RAID60 (so I could loose 2 drives and still get a performance boost of striping ie. 60)

I have began reading a lot about ZFS; not I am totally confused as to what is the best route to go with FreeNAS.

Do I use the Adaptec controller and setup RAID 60, then install FreeNAS and will it then see the RAID60 as one drive ???


Do I disable the RAID of the Adaptec card, then install FreeNAS and have it use them as ZFS?

Really could use some educated guidance.....

For completeness, the two XenServer servers are identical hardware: GigaByte 970A-D3 MB, AMD 8150 8 core 4.0 Ghz CPU with 32 GB 1600 DDR3, 1 - 240 GB SSD S3 for OS, Intel DUO GB NIC (for iSCSI), DVDRW.
Question by:sdragoninc
  • 3
LVL 47

Accepted Solution

dlethe earned 500 total points
Comment Utility
ZFS will provide better overall performance in most situations, and certainly higher data integrity.   It isn't fair comparing HW raid controller with ZFS.

The big reason is ZFS can use as much RAM as you have to prevent the need for some I/O.  It uses ram for read cache.  It can use a SSD for read or write cache. So if you want 100GB of read and/or write cache then you can do it.  Most cache you could get with the adaptec is a few MB.

But so much more to it...   You can tell ZFS to do hot snapshot backups, or keep an additional copy automatically of files in a certain directory.  Do that, and now you have more read load  balancing.  (Data in 2 places, typically different disks so when you read, it can give you the data for those files on whatever disk has least traffic at that instant)

You can do hot snapshot and online expansion in ZFS of just a filesystem.  You can't do that in adaptec hw raid.

The money you save going with a JBOD controller can be spent on more RAM and/or a small SSD for read cache or a ZIL (write intent log, or even a mirrored ZIL).

Bottom line, the Adaptec card will hurt performance if it is in any kind of RAID mode.  You must expose individual non-RAID disk drives.  If that card is not capable of presenting the Seagate, WD or whatever disks you have as n individual disk drives, then don't use the card. It will slow you down. The dumber the controller the better when using ZFS.
LVL 21

Expert Comment

Comment Utility
In short, you should not use HW raid on top of SW raid, it should be one way or the other. You should use the SW raid provided by freenas for performance and scalability purpose.
If there is a drive failed, zfs have the ability to rebuild the array independently. If you need to move to different piece of hardware, you can export and import volumes.

I suggest you to take some time and read through the freenas hardware guide -
LVL 47

Expert Comment

Comment Utility
ZFS is uniquely intolerant to even using HW raid in any way.   ZFS does a flush-on-write.  It does so for data integrity and to get around the RAID5 write hole.  (google that for an explanation).

So with the flush on write, this will kill hardware RAID performance, even if you just limit hardware RAID to RAID1.

Author Closing Comment

Comment Utility
Ended up using ZFS2 and disabling the Adaptec RAID functions.  Had I known prior to purchasing, I would have bought a simpler SATA expansion board instead of RAID. Lesson learned.
LVL 47

Expert Comment

Comment Utility
You might want to look deeper.   I do not know off top of my head if the Adaptec controller ends up virtualizing single disks.  It may do so.

Do the drives appear to ZFS as whatever make/model they are supposed to be, with the full number of blocks as documented in the specs?  (You can run the solaris format command to query such information.   It won't format the disks unless you specifically tell it to,  so select the drive and look at the # of blocks & make/model)

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

Suggested Solutions

this article is a guided solution for most of the common server issues in server hardware tasks we are facing in our routine job works. the topics in the following article covered are, 1) dell hardware raidlevel (Perc) 2) adding HDD 3) how t…
Learn about cloud computing and its benefits for small business owners.
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now