[2 days left] What’s wrong with your cloud strategy? Learn why multicloud solutions matter with Nimble Storage.Register Now


Storage and Backup Solution for Video Editing System

Posted on 2014-01-11
Medium Priority
Last Modified: 2016-10-27
Hi Experts,

I been doing some research on the best storage and backup solution for my needs and wanted to seek an experts opinion before I spent my $$.

Up until now I've been backing my system up on an external hard drive, which has recently failed and as I am reaching near capacity with some of my internal drives I'm contemplating a few different options relating to expansion and backup...

Note: I am an amateur when it comes to all of this so please excuse any inaccuracies or lack of knowledge with the information below.  

I'm a videographer and amateur photographer. I work with large HD video files (~100mb-1.2GB) and RAW photography files (~18mb-50mb).

Current Storage Set Up  
Internal 1 [60% full] - 1x Crucial M4 256GB SSD
(This is my dedicated Operating System Hard drive)

Internal 2 (Set up in RAID 1) [95% full] - 2 x 1TB SATA 7200RPM 32M
(This drive is used for photos, completed video projects, documents, music etc)

Internal 3 (set up in RAID 0) [35% full] - 2 x Hitachi 2TB Deskstar 7K3000 SATA 6GB 7200RPM / 64MB
(This set up is used purely for rendering and editing large HD Video Files, I'm also using to store video files*)

*No redundancy for this set up - hence I need to find a solution ASAP!

NOTE: I use Acronis software for backing up.

Possible Solutions
1. Buy two external hard drives. Use one for full system back up. Use the other to store additional photos, completed projects, documents etc
Pros: Affordable, Easy to set up
Cons: Not so 'neat' having several external hard drives floating around, not convenient having files stored in separate locations.

2. Buy a new external hard drive for full system back up and take 'Internal 2' out of RAID 1 to create more space for completed projects, photos etc
Pros: Affordable, fairly easy to set up
Cons: Less redundancy, will most likely need more space in 12months time.

3. Buy a NAS and set up in RAID 5 (start with 3 x 2TB drives). Move all of my docs, completed projects, music and photos to the NAS. Use Internal 2 as full system back up (could also perform full system back up on NAS) OR take 'Internal 2' out of RAID 1 and turn into RAID 0 for additional storage for Video editing files and possible editing of RAW photo files.
Pros: Plenty of space for future, Good redundancy
Cons: $$$

4. ???

My Budget?
I would be in a state of despair if I lost all of my files so it's hard to put a price on that.
At this stage if I could find a solution for around ~$500 then I would be happy. However if spending a little more meant meeting my needs for now and into the future then I would consider doing so.  

Current System
PSU - Corsair HX-1050 1050W ATX Modular Power Supply
M/B - Asus PX79 Pro
CPU - Intel CORE i7 3820/3.60GHz
GPU - Asus GTX660 TI-DC2-2GD5 2GB GDDR5
RAM - G Skill 16G(4X4G) DDR3 2133MHZ
Case - Antec Twelve Hundred (V3) Full Tower
O/S - Windows 7 Pro (64)
Question by:BPro2008
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 4
  • 3
  • +1
LVL 47

Accepted Solution

David earned 350 total points
ID: 39774243
None of the above.  Use a network / internet based backup solution for offsite backup.  Many are out there.  Google for internet backup services and do some research.  They all have strengths and weaknesses for whatever your budget and patience allow.

I will also say RAID0 is totally unacceptable.  Were you aware that windows native software RAID1 does read-load balancing?  100% of all read I/O in RAID1 is twice that of a single disk drive in a perfect world.  Writes are no worse than a single drive.

So use a small SSD for rendering as the temp device, then copy the finished project to a RAID1 somewhere.

RAID1 is more than protection against a HDD failure. It protects against bad blocks and data corruption.

P.S. you have backup & archiving.  People need both.  You have no protection against fire or natural disasters with your setup. You certainly have no protection against human error like copying files in the wrong direction or even windows bugs or a virus.

If the data is important use offsite backup.  Copy files to free space in another RAID1 drive set if you want a local backup.

Author Comment

ID: 39774296
Hi dlethe,

Thanks for your response,

I have a very limited monthly data allowance (4GB) with my current internet service provider (due to my location), so internet backup services are probably not going to be suitable for me at this point in time given the upload and downloads that would be required?

Which means for the time being I will have to accept the risks of loss of data due to natural disasters or theft. Although I could use an external hard drive to perform backups and store off site in a secure location (a degree of human error would still be an issue here though).

You certainly have no protection against human error like copying files in the wrong direction or even windows bugs or a virus.

Q - not sure what you mean by 'human error like copying files in the wrong direction' could you explain this further?

Q - As the for bugs and viruses, I have anti virus software and have my OS on a separate drive so should something fail and I can re-install windows without disrupting other hard drives. I realise this is not full proof, but would it not offer some protection? Apologies if this is a really naive view...

I will also say RAID0 is totally unacceptable
Why is that? Are files not read faster in this set up?

Were you aware that windows native software RAID1 does read-load balancing?  100% of all read I/O in RAID1 is twice that of a single disk drive in a perfect world.  Writes are no worse than a single drive.
No, can you explain this further? Sorry, but this is beyond my understanding...

With internet back up service potentially on hold I will look to local back up.

So use a small SSD for rendering as the temp device, then copy the finished project to a RAID1 somewhere.

My Motherboard only has one dedicated internal port connection for SSD drive and as this is already being used would you suggest I get an external SSD drive for rendering? Sounds like a stupid question but just double checking.

Copy files to free space in another RAID1 drive set if you want a local backup.

I don't have much more room in my computer for more drives (i.e. a RAID1 set up) and if I change the current RAID 0 to RAID 1 I will lose 2TB of data storage. Looking purely at a local backup (and storage), how would you recommend I proceed?

Thanks for your time dlethe, really appreciate it.

Assisted Solution

insidetech earned 300 total points
ID: 39774369
OK... So yo have some options because you do have some awesome hardware.
For starters let's figure out your priorities though.

1. Boot drive is SSD ... That is great, you have fast boot and your programs load fast.
2. Internal 2 is 1 TB RAID 1 Mirror set. Performance not important...
3 internal 3 is 4TB RAID 0 stripe.... For performance... If you did set up the RAID1 on the controller and not in the OS the performance is really there despite what other "experts" say.
Problem you have is that as one drive goes, so does the entire set and 4 TB restore will take a while.
You may not know this, but your MB has two special ports for SSD cached HD.
If you do have the resources, purchase the biggest/fastest SATA Drive with complementary SSD for the cache and you will have a supper fast drive for the rendering functions.
After that, you can re-assign the RAID 0 drives for a data backup use.
Also your MB has eSATA ports so you can attach an external drive with build in RAID. Look at Vantek NextStar MX products.
Look at Acronis products for an excellent software back up solution.
Hope that this helps.
Nothing ever in the clear!

This technical paper will help you implement VMware’s VM encryption as well as implement Veeam encryption which together will achieve the nothing ever in the clear goal. If a bad guy steals VMs, backups or traffic they get nothing.


Author Comment

ID: 39774545
Hi insidetech,

Thanks for your help!

From what I can see on the MB I only have 1x6Gb sata port free.

6Gb/s Sata Port 1 - Free
6Gb/s Sata Port 2 - SSD drive
3Gb/s Sata Port 1 - DVD Burner
3Gb/s Sata Port 2 - DVD Burner
3Gb/s Sata Port 3 - 1TB in RAID 1
3Gb/s Sata Port 4 - 1TB in RAID 1
6Gb/s Sata Port 1 (Cache Port) - 2TB in RAID 0
6Gb/s Sata Port 2 (Cache Port) - 2TB in RAID 0

LVL 88

Assisted Solution

rindi earned 350 total points
ID: 39774586
Buy several disks to backup to, backups should never be done to just one location. You should rather cycle between them. Also always remove the backup disk from the system after it has completed, and then store it off-site.

You can buy standard HD's for your backups, and additionally get a USB dock to drop them in. That keeps the price lower. The first link is to a USB 3 dock, but there are also cheaper USB 2 docks:

LVL 47

Expert Comment

ID: 39774800
Here is the deal with RAID1 when using the native software RAID in Windows.
All data is in two places at once, right?  Well the O/S knows this. The O/S also knows what I/Orequests are sent to each drive, and which I/Os are pending at any moment.

So 100% of the time, the O/S can choose what disk can get the data faster.  In a perfect world, on reads, each disk gets half of the I/O, meaning you get 2X the performance.

But in RAID0, at best, only 50% of the data is on any one drive.  My over simplified summary is that if a program is using RAID0 then it has to wait for the disk that has the data it needs.  In RAID1 it never has to wait, because the data is in two places at once.

So if you want READ performance, regardless of the type of I/O use RAID1.  

Write intensive stuff should be the SSD, or RAID0, but in grand scheme of things you may find that what you think is write intensive also is read intensive, so RAID1 still may be a better deal.

Note: You better be using the windows software RAID, not any cheap fake raid on a motherboard. They do NOT do read balancing.   Expensive RAID controllers do load balancing.

What is make/model of your RAID controller (or are you already setting up AHCI & windows dynamic disks to do the RAID?)

P.S.   With windows software RAID, you can partition the same disk to do half RAID1, half RAID0, by the way. Just create 2 partitions.  You will still get the RAID1  & RAID0 benefits, but it just won't be as good as if the entire disk was either RAID0 or RAID1.

My deal with human error comment.   If the backup is J:  and source is C:\   maybe you carelessly "backup" the data from J:\ to c:\   by mistake.   Such things happen to the best of us.

Or maybe you fat finger windows explorer and drag something the wrong place or way with the mouse and can't Kill the process fast enough.

Expert Comment

ID: 39774949
This is what you have... Note the two eSATA (external ports)  Also it is Vantec.com for good quality external storage enclosures.

Intel® X79 chipset :
2 x SATA 6Gb/s port(s), white
4 x SATA 3Gb/s port(s), blue
Support Raid 0, 1, 5, 10
Marvell® PCIe 9128 controller :
2 x SATA 6Gb/s port(s), white
ASMedia® ASM1061 controller :
2 x Power eSATA 6Gb/s port(s), green

Expert Comment

ID: 39774956
For your transition in moving your data unplug one of your CD drives to free up a port.
Forget about the "academics" RAID on Asus hardware works great as long as you know the downside. Also life is too short to move data over USB. ;-)
LVL 47

Expert Comment

ID: 39774964
That Marvell controller does not do read load balancing.  It does not have cache for buffering I/Os. You'll get much better performance on your RAID1 if you don't use this controller for it.  

If you use this controller for your RAID0, then it will be pretty much a wash in terms of performance if you let this controller do the RAID0.

There is no difference in the 3Gbit or 6Gbit ports, since the best any of your HDDs could do is around 2Gbit/sec, and even then it would be for only a few milliseconds on re-reads.

Author Comment

ID: 39775644
dlethe - Yes, I'm using Windows RAID software instead of the Marvell controller on the MB as I was informed that this would be a better solution. Probably because of the point you mentioned in your post above. I don't have a dedicated RAID controller card at this point in time.

Insidetech - Thanks for this. However I don't want to unplug one of the DVD drives, unless you were referring to this as a short term solution?

The Vantex external cases look good and is possible solution. Although I just feel that if I'm going to start building outside my case then I should build a NAS or at least something that is scalable and offers additional uses such as multiple user connectivity. IF I built a NAS though I'm under the impression that performing heavy editing tasks with large I/O will be bottlenecked through the connection from my computer to the NAS (via network cable)? Is this correct? So it would best served for other files not used for video editing and thus freeing up space inside my machine on HDs used for video editing?

dlethe - your simple explanation on RAID1 and RAID0 makes sense to me. However in the link below it would suggest otherwise. Does this have anything to do with running RAID through windows O/S vs a dedicated RAID controller? Or are they wrong?

In the article they suggest to use RAID 3 for video editing - I'm under the impression I would need a dedicated RAID controller for this?

RAID for Video Editing - Adobe Community

Rendering on SSD - Quite a few people on forums have reported that continuous writes on an SSD drive will diminish it's life very quickly and that you're not using it to its full potential. What's your take on this? Because I like the idea of having a dedicated SSD drive for rendering files..

What about...
Assuming I cant set up RAID 3 through Windows O/S, what about getting 2 more 2TB drives (removing the 1TB drives) and setting up RAID 10 - use for source files (photo & captured video)

Installing a single drive (SSD/SAS) in the free 6GB slot - use for rendering, preview files, media cache

Building a 4-bay NAS (buy 2 more 1TB drives) and set up in RAID 5 - music, movies, docs etc / put the 2 existing 1 TB drives in a Vantec external case.

Back up to the cloud when I improve my internet connection.
LVL 47

Expert Comment

ID: 39775690
The article is profoundly dated, incomplete, makes flawed generalizations, and is downright wrong on several points.

I know quite a bit about SSD wear leveling as I have developer NDAs with many of the SSD manufacturers and write software to measure and report such things.

Don't worry about "burning up" a modern SSD.  I've run tests for military projects where we ran 100% writes, CONTINUOUSLY, in ovens, for over 6 months on hundreds of SSDs of various makes and models.  Many survived that process.  Now, of course we didn't use $89.95 consumer drives, but we didn't use MIL-spec drives either for our assessment. We used commercial class off-the shelf SSDs.

Just do rendering on a SSD, the best you can afford, minimum size to do the largest project you could possibly need.   Also max out the RAM.   Some SSDs can do 100,000 random IOPs and 550+ MB/sec sustained.  Others do 5,000 random IOPs and 240MB/sec sustained.   Both do a heck of a lot better than 100 random IOPS and 120MB/sec sustained like the typical SATA-3 HDD.

Author Comment

ID: 39775812
Brilliant, I was hoping you would say that.

Which one..
SSD - Corsair 120GB Neutron SSD GTXB with the state of the art LAMD LM87800 processor. R/W: 555/330 MB/s VS Corsair 128GB 2.5" Force Series GS SSD 560MB/s Read 535MB/s Write, fast Toggle NAND

I just need to finalise a solution for additional space (with redundancy). Because if I now convert my 'Internal 3' set up from RAID0 to RAID1 I'll be down 2TB putting my use for this set up close 70%.

Replace the 1TB hard drives?
LVL 88

Expert Comment

ID: 39775818
You had asked about backup. A good, reliable backup needs to be rotated to different media, so you have more than one backup, and you should be able to store those media offsite. For that several single disks are better suited than a NAS. A NAS could be an option if it were in a geographically different location than your PC, so it would also be offsite, which is unlikely in your case because of the limited bandwidth you mentioned. If something happens to your office, your live data and the backup would likely be gone.

Author Comment

ID: 39775996
Thanks Rindi that's good advice. I didn't meant to neglect your previous post, my main priority at the moment is finding a suitable storage solution (ideally with local redundancy). Second to that is finding a better back up solution, which you've identified above and dlethe identified with online back up.

If I removed the 1TB drives I could put them in an external (Vantec) case and use them as means of backing up the system and then storing the drives off site? As you mentioned I should rotate to different media. But that does require a lot more effort than I'm willing to take...at the moment.

Author Closing Comment

ID: 39781948
Some good advice, but I'm still wondering what would be a suitable long term storage solution for my needs.

The 'Back up' solution has been answered 100%. But not the storage.

Featured Post

NEW Veeam Agent for Microsoft Windows

Backup and recover physical and cloud-based servers and workstations, as well as endpoint devices that belong to remote users. Avoid downtime and data loss quickly and easily for Windows-based physical or public cloud-based workloads!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article we will learn how to backup a VMware farm using Nakivo Backup & Replication. In this tutorial we will install the software on a Windows 2012 R2 Server.
Your data is at risk. Probably more today that at any other time in history. There are simply more people with more access to the Web with bad intentions.
This tutorial will walk an individual through configuring a drive on a Windows Server 2008 to perform shadow copies in order to quickly recover deleted files and folders. Click on Start and then select Computer to view the available drives on the se…
This tutorial will show how to configure a new Backup Exec 2012 server and move an existing database to that server with the use of the BEUtility. Install Backup Exec 2012 on the new server and apply all of the latest hotfixes and service packs. The…

656 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question