RAID configuration for growing audio/visual production company

Hello -- I operate an audio/visual event services company which provides video production, editing and storage for events of all types.  

We're utilizing (2) PC-based video switchers that record video files directly to 3.5" HDDs.  Usually a single event requires at least 1 TB of data.  

As we have only been providing this service in this manner for a couple of years, we have been storing the data on the HDDs themselves, purchasing refurbished 3 TB drives from one of our vendors so that we continually have enough space to store show footage.  

What I would like to accomplish is the following:

A RAID system that allows us to utilize the (21) drives we currently have as one drive, accessible via in-house network.  This would allow us to simply connect the video switchers to the network to store footage, and not have to shift actual drives.  

Also, we would love to be able to edit directly off the network; i.e., our video editor would be able to simply work on a project by connecting (likely hard-wired) to the network in-house without having to offload footage to an external USB drive.

I need direct instruction on this.  I am skilled in many facets of IT but have never built out a RAID server before.  Our entire network consists of PCs with few Macs involved.
livewiredjAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

David Johnson, CD, MVPRetiredCommented:
you will need an enclosure that can hold all 21 drives to show them as 1 drive letter plus something like windows storage server 2012R2.

Buying refurbished consumer drives is highly not recommended for an enterprise level system. No Commercial NAS will support these desktop drives you need enterprise class drives. Don't even think of using RAID 5 since you are well above the limitation of drive technology and raid 5's capabilities.

The content of your question raises many concerns i.e. BACKUP and other significant issues that you've not addressed.  You could use something like freenas.  I am also under the impression that these drives are SATA and not SAS drives.
Carlos IjalbaIT Systems DirectorCommented:
Well, the problem that you also have is that when you convert "normal" drives to RAID protection (whichever flavor) all the drives involved in the RAID group need to be formatted, so you loose all your data.
And I suppose that you don't want this to happen...

But as David has recommended, the use of refurbished drives is a problem waiting to happen. It all really depends on the footage that you are storing: if it is a customer's wedding from a couple of years ago, it probably won't matter much, but if it is some event like a conference or a concert that you want or need to keep for long time retention, then you will better use a system that uses a storage subsystem with RAID protection, backup to tape (LTO6-LTO7 recommended) and that supports data archiving as well as backups, so you don't need to have all the storage online.
Tarmo KabonenIT&Helpdesk ManagerCommented:
Did You already moved to some kind of NAS/Storage/Server solution? I've tested MS 2016 with deduplication of different multimedia and results are actually quite amazing. With "reasonable" costs HPE MSA extendable entry level storages are also possibility. And if it is a very low budget need, then Amazon Cloud Unlimited storage plan is what You really need keeping offline those media, that is really used.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
skullnobrainsCommented:
you can build a nas.
this would require a big enough case and a number of raid cards. ( given the price of such cards, you may consider buying bigger drives and less cards at this point )
using raid 10 should be your best bet
if this is freenas, use zfs and you'll have the additional benefit of proper data consistency and the possibility to add extra pairs of drives without reformatting the whole thing. this can also be very helpful if some of the disks hold data that you need while you're migrating.

you can also use a bunch of ( possibly existing ) computers and a distributed filesystem such as glusterfs. the basic setup would store 2 copies of whatever you write on 2 different hosts. files are kept in a single place so grabbing stuff from a dead cluster is feasible if needed. adding pairs of hosts is feasible but more complicated than adding drives to freenas.

--

i totally second using lots of SATA drives rather than few expensive SAS drives for both setups. but you had better use new drives, forget about 5400rpm ( not much less expensive, much slower, less durable... ) and prefer hitachi to maxtor unless you really want to spend your time changing disks.

also note that buying 4 10Tb drives, plugging them into an existing box without a raid card, and installing freenas should get you running in no time and with little chances to break something. this probably should be considered : it is less expensive than buying a bunch of raid cards and much simpler to setup than either of the above.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
MultiMedia Applications

From novice to tech pro — start learning today.