NVR RAID Read Performance

I have a Areca ARC-1883ix-24 RAID card with 2 Gigs of Cache. I am using it in a RAID 5 configuration on a NVR system running Avigilon Control Center (NVR Server Software).
I am having issues with playing footage in reverse although things are working fine otherwise.
This is a personal build of the server. I did this because a server from Avigilon cost somewhere in the 50k range and I built this for about 3k
I am using 8 4GB 6Gbs dirves at 7.2k speed.

I am curious about improving read performance of the array.
Will the Cache help? ie. Write Back.
Is it worth upgrading to 8 GB of cache if the above is true?
Is there anything else I can do other than running a RAID 0 setup? Although I might consider doing a test with RAID 0 to verify the system is capable of playing footage in reverse without issue.
Will upgrading to 12Gbs drives help this issue?
Could the back plane be causing issues? (the case is a piece of junk and has a built in back plane. Norco RPC-4216)
Is there anything else you can recommend to help this issue?

Thanks,
Stuart
LVL 2
beatifiedAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

andyalderSaggar maker's framemakerCommented:
More read-ahead cache could help normally by telling the disks to get a few more blocks after the one asked for, but playing backwards means wanting the previous blocks and I doubt the controller would think to do that. You probably would be better off using SSDs.
0
CompProbSolvCommented:
Have you done any basic disk speed tests to see what sort of performance you are getting?  HDTune is a common (free) one.

While RAID 0 would be faster, you'd have no fault tolerance.  I'd look at RAID 01, or RAID 10 if performance is the issue, though that would change your capacity from 28G to 16G.  RAID 5 is generally discouraged these days.

Are those really 8G or 8T drives?
0
dfkeCommented:
Hi,

if you insist on keeping raid 5 the only option is to add more spindles ie. disks in the raid 5 set. This will speed up your read performance but write performance will not increase.

Cheers
0
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

beatifiedAuthor Commented:
@Handy Holder
Yeah I figured that was why I was having issues to begin with is that the data is being accessed in reverse of what is normal. I cant personally think of any other situation that something wants to read a file backwards except in playing a video backwards witch is still pretty unusually except in DVR/NVR's. Unfortunatly SSD's are out of the question simply because of the drive size. I currently have a 24TB array it will be getting larger in a little while and doing that with SSD's would be pretty tough.

@CompProbSolv
I haven't done any tests yet but I will soon and get back to you. I agree that a 10 or 01 would be best and I can consider that over the long term but I dont think it should be needed as the Avigilon Servers use RAID 6 as seen here. And no they are not 8GB or 8TB they are 4TB and when I stated 8GB I was referring to the RAID controller cache it currently has 2GB.
Can you please explain why RAID 5 is discouraged my guess would be a combination of speed and reliability. The problem there would be that with 24 drive capability on the controller I would max out at 120TB with 10TB drives and RAID10 or 01 but with a RAID 5 or 6 I would be around 230TB or 220TB and although it seems like why do I need that much we have pretty much solely 5MP camera's running at 36Mbps and will eventually have about 35 to 40 of them.

@dfke I like that idea hadn't thought of it that simply but maybeing trying my current config in a RAID 0 to see if there is any real benifit first then if I do see better read preformance and improved play in reverse I will do it that way that is a really simple solution to the issue and can also increase storage.
0
beatifiedAuthor Commented:
Here are the results from the benchmark. I started the test with the NVR Server software shut down and then ended with it running. So that is the performance drop you see.

Anyone have any input on me upgrading to 12Gbs drives I would think that would help immensely.

Just an interesting fact my NVMe drive is running at 1250MBps pretty crazy it'd be great to have an array of those but then you'd probably hit the limit of the bus.
HDTune.png
0
andyalderSaggar maker's framemakerCommented:
How about copying the file to a stand-alone SSD then reading it backwards?

RAID 5 is discouraged with high capacity drives because there's quite likely to be unflagged bad blocks on high capacity drives. If you lose a disk and also have a bad block on another one it can't rebuild. That's why double parity is recommended, lose a disk with that and it can get past the bad block.

RAID 10 doesn't help sequential read speed much over RAID 0, since you can get the data from two disks random read is faster because one disk can get one block and the other get another. but sequential not much difference in performance since one disk is already atr the data so just has to keep on reading (unless it's backwards). RAID 0 is of course discouraged as one dead disk kills it all.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
beatifiedAuthor Commented:
@Handy Holder Maybe I can try a backup of some footage on an NVMe drive to start with and see if I have decent rewind capability.
0
andyalderSaggar maker's framemakerCommented:
Just seen your second comment, 12Gb bus speed won't make any difference, it's the mechanics that's the bottleneck. You can't get an array of NVMe disks except using software, hardware controllers can't use them as they need PCIe connectivity to each SSD.
0
beatifiedAuthor Commented:
So I figured out that in reverse it is only playing at 1 fps which I'm sure if for performance. In fact I bet it is just playing the key frames. And it seems to be playing smoothly at that frame rate. Maybe I'm not having the problem I though I was?
1
beatifiedAuthor Commented:
@Handy Holder Thats true hadn't thought that far into it.
0
beatifiedAuthor Commented:
Handy Holder's comments led me to the trouble shooting that led to the answer but dfke had a great point in more drive to increase read performance.
0
Adam BehrendtSenior System EngineerCommented:
@beatdield you hit it on the head.  It is just playing key frames.  It is disk performance related but also you should do playback remotely not on the server.  The client is able to help smooth that out, and you probably see a message about no graphics cars when you open ACC if on a server.
0
CompProbSolvCommented:
@Handy:
Actually, there are now a number of NVMe RAID controllers.  I've not used any yet but am very interested.  For example:

http://www.highpoint-tech.com/USA_new/CS-product_nvme.htm
0
andyalderSaggar maker's framemakerCommented:
@CompProbSolv, They're not really RAID controllers, they pass the PCIe lanes from the NVMe devices straight through to the CPU which does all the work. If you don't have the software installed each drive appears as a separate device.
0
CompProbSolvCommented:
@Handy
Thanks.... I didn't realize that.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Server Hardware

From novice to tech pro — start learning today.