Caching on an Adaptec Raid controller with SSDs

I had an interesting little romp through raid today. I wanted to benchmark a Raid 5 array with Cheetah 15k drives vs Intel Pro SSD drives.

   The Cheetah 15k SAS drives were on a Adaptec 6405 controller and the Intel Pro SSD drivers were on an Adaptec 8405 controller. HD Tune benched them at about the same. Average transfer 300MB/s. Then I remembered as I was setting up the SSD array the Adaptec Firmware came back with the message that "we don't recommend enabling read or write cache with SSD drives" so I disabled. I went back in to the controller and enabled both the read and write cache. Lo and behold the average transfer speed doubled to 626MB/s.

   So ... anyone have any idea why Adaptec wouldn't recommend the read or write cache with SSD drives? Wow did it make a difference!
LVL 15
LockDown32OwnerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Thomas RushCommented:
The cache on the RAID controller is architected to improve performance of hard drives.  It is built 'good enough' to make a huge difference in hard drive performance, but building it to make a significant difference in SSD performance would add tremendous cost at the HW level.

Second, reading disk blocks into RAM to avoid the overhead of having to read those blocks from disk a second time makes sense -- but SSDs don't have the same kind of overhead for seeks... and the caching and cache management   add overhead themselves -- so small an amount that we gladly pay it for HD arrays, but too much to pay when working with SSD arrays.

Also consider optimization tricks used with hard drives such as track read-ahead, that will do nothing good for SSDs, and which may even decrease application performance!
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
DavidPresidentCommented:
First the Adaptec is just not architected for SSDs.  It will severely limit performance.    Secondly, read the specs on the SSD.  Some have a non-volatile write cache.   (But not the low-end ones).

Finally, if you are doing SSD for performance, then don't just KILL bandwidth and IOPS by going RAID5.  Do software-based RAID10, and make darned sure you set the alignment right.   One can easily get over a million IOPs or > 1 GBYTE/sec running with software based RAID10 with 4 decent SSDs with a non-RAID PCIe 3.0 LSI SAS HBA.  You can get > 2GBytes sec throughput if you add a few more SSDs.
0
LockDown32OwnerAuthor Commented:
Thanks guys but we went a little off-topic. The cache on the Raid Controller did make a BIG difference. Hence the question why would Adaptec not recommend enabling the cache on the controller when it does make such a HUGE difference?

   @dlethe who's controller is architected for SSDs and have you ever benched the average transfer? You think it would be a lot better them 600MB/s?
0
Get expert help—faster!

Need expert help—fast? Use the Help Bell for personalized assistance getting answers to your important questions.

DavidPresidentCommented:
http://www.avagotech.com/products/server-storage/host-bus-adapters/

The 9300 family will easily do > 2GBytes/sec and over 1M IOPs  with 6 of the 12G HGST SAS SSDs in a RAID10 config if your motherboard is PCIe 3.0

Otherwise, go with a 9200 family and use some of the 6Gbit SAS HBAs, but you'll need more than 6 SSDs to get 2Gbytes/sec if you want that much throughput.

Also, avoid SATA SSDs if you care about performance.  

The write cache in the controller has nothing to do with enabling write cache in the SSDs, but real world, it won't make much of a difference as long as you have a UPS on the computer.  It will take worst case a few hundred milliseconds to flush any I/O to the SSD even if the cache is completely full of information to write.  If you have no UPS on your computer, then you are exposed, but if your SSDs have non-volatile write cache, then odds are nearly zero you risk data loss in event of power loss.

Adaptec is marketing to people who by crappy consumer class SSD products, so they're being conservative and assuming you have slow SSDs with volatile cache, and no UPS.  While the Intel Pro is STILL consumer class, it is top-of-the-line consumer class, so it can flush pretty quickly.  Real world, don't worry about it, and back up.  RAID5 is no substitute for failing to back up.
0
LockDown32OwnerAuthor Commented:
Correct the write cache on the controller has nothing to do with the write cache on the SSD but that is what I am saying. The cache on the controller did make a big difference. If you have a UPS on the computer then you can enable both the controller write cache and the drive write cache. One alternative is a battery backup on the controller.

Anyway... the question really still remains and I want to focus. Adaptec says not to enable the controller cache if you are using SSDs yet it made a huge difference. Why do you think Adaptec wold say something like that when it made such a big difference?

Did Avago buy out LSI?
0
DavidPresidentCommented:
Of course the cache on the controller makes a big deal because this allows it to read queued I/O from other SSDs before flushing writes.  It facilitates a queue.  Without write cache then the system can't process until data is flushed to the disk.

(Adaptec, as I said, markets to consumers, they said this to minimize calls from people complaining about data loss).

Yes Avago bought LSI.
0
DavidPresidentCommented:
Or, to be blunt ... Adaptec knows that anybody foolish enough to build a RAID5 with SATA SSDs on this controller clearly doesn't know what they are doing, so they need a message such as this.  Furthermore, the controller has no idea if you have a UPS attached to protect the adapter's cache.

Other adapters have a battery protected write cache too, for what it is worth.
0
LockDown32OwnerAuthor Commented:
Interesting comment about Adaptec marketing to consumers. I have used Adaptec in servers since they first came out. Throughout the years I have looked at LSI on several occasions but have just never found a reason to switch. Came close a couple times just never did it. The reviews on Newegg were never really good enough to get me to jump ship. So you really like LSI huh?

   What about hard drives. What is your preference there?
0
LockDown32OwnerAuthor Commented:
"Or, to be blunt ... Adaptec knows that anybody foolish enough to build a RAID5 with SATA SSDs on this controller clearly doesn't know what they are doing". On what controller and why would that be foolish?

Adaptec does have flash backups for all their controllers as mentioned above. That it what I meant when I said battery backup for the controller.
0
DavidPresidentCommented:
None of the big boys use adaptec SAS/SATA HBAs.  EMC, IBM, NetApp, ... all LSI..  LSI owns the patents.  Adaptec is years behind.
0
DavidPresidentCommented:
Why would it be foolish?  Simple.  Unless this is an offshore oil rig where you must run SSDs due to G forces, then the benefit of SSD is 100X-10000X more IOPs than rust disks.  Go software RAID10 or RAID1 and at very least you double the read IOPs.  Go RAID5 with hardware or software RAID, and your IOPs decrease dramatically.

To make matters worse, this card destroys throughput because all the bandwidth is restricted to the internal PCI lane of the controller.  Heck, this controller will be the bottleneck if you just have one enterprise class HGST SSDs.
0
LockDown32OwnerAuthor Commented:
I took a look at the prices on some of those HGST SAS SSD drives. We are back at the good old bang for the buck scenario. If prices were no object....

   Unfortunately I deal with small businesses where price is an object and even today Raid 5 still gives you the most usable space with the least amount of drives and yet has fault tolerance. Software Raid has never been a good idea. I wish the sky was the limit but it isn't. I'll look again at Avago. I did actually try an Intel a month or so ago (it was an LSI). Never opposed to trying new things.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Server Hardware

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.