NAS & LSI Megaraid

Hi All,

We've had a requirement for a NAS fairly recently so I assembled some parts and came up with a NAS running Openfiler 2.3 (Redhat based)

The object of this unit is to run NFS share for a couple of vmware servers and a backup solution for our other servers.

So the Problem:   Raid 5 write speed!!
The Servers Hardware consists of
8GB of Ram
TST ESR-208 SAS/SATA 2U Case, 8x Hot swap HDD, 600W Power Supply,
8 x Seagate SATAII NCQ 1.5TB 7200RPM 32mb Cache 10TB (Raid5)
1 x 320Gb SATA 2.5" HDD (os drive)
exFujitsu LSI Logic / Symbios Logic MegaRAID SAS 1078 (rev 03)

Simple DD test I'm only getting around
sync ; date ; time dd if=/dev/zero of=bigfile bs=1M count=10000 ; date ; time sync ; date
Wed Apr 21 19:53:26 EST 2010
10000+0 records in
10000+0 records out

real    3m26.104s
user    0m0.008s
sys     0m12.957s
Wed Apr 21 19:56:52 EST 2010

real    0m18.624s
user    0m0.004s
sys     0m0.152s
Wed Apr 21 19:57:11 EST 2010

So around 48mb/s, Write.

So the issue I'm having is i'm only able to obtain  30-50mb/s Write speed
in the array however easily obtaining over 500+Megabytes/s read.

I understand raid 5 takes a pretty big performance hit however i would have expected over 100mb/s.

I Currently have write-back caching enabled on the raid volume.

Any ideas?


Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

what did you set as the block size?   Are you using md-based software RAID or hardware RAID?  What firmware on the LSI card?   Was the dd tst run on the local system with the RAID, or across a network?
qtenAuthor Commented:
Hi dlethe,

The LSI Megaraid card is a hardware only raid device which dosn't support JBOD so i wasn't able to test the software raid as an option.

Blocksize of the Raid 5 is 64kbyte. I have done a little research on increasing the blocksize however nothing to say this is the issue.

The dd test was completed locally on the server. as was the read test of 500+mb/s.

Firmware on the card 2007-11-19 I believe.


Actually, I have maybe 8 different models of LSI SAS/SATA controllers, including the 6Gbit SAS-2 models, and some can run RAID-only, others can only be non-RAID, others go both ways, depending on the firmware (-IT or -IR) but moot point now that I know you are using one that is hardware RAID.

A few things from the top
 * Upgrade firmware & BIOS
 * Look at the individual HDD settings to see if NCQ is enabled on the drives.
 * Is this a VM? Please clarify, if so, it changes everything
 * Re-run benchmark from single user mode, this minimizes overhead from other jobs
 * What are your FILESYSTEM parameters?   Please report them. Are you running ext3?
 * run the tune2fs program, I bet you can make a major improvement here.  
 * turn off journaling, you do NOT want to be journaling on this particular LUN with such a large stripe size. It will hurt your I/O.
 * There is a lot of tuning you can do with LINUX as well, regardless of the controller.      
 * WHat kernel are you running?

There are just so many things to look at, here is a nice doc from LSI

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
qtenAuthor Commented:
Resolution was found via dlethe's comments.
qtenAuthor Commented:
Got me on the right track to resolving the issue.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.