• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 180
  • Last Modified:

How to store, serve, and back up 100 TB of PDF files?

My organization currently stores, serves, and backs up about 4 TB of data. Now, out of necessity, we are converting all of our microfilm to PDF files. A vendor is doing that part, and they estimate that once converted, there will be 100TB of PDF files. I don't even know where to begin with managing that much data.

The backup medium should have a long shelf life. Otherwise, we'll be constantly re-backing up the data as the expected shelf life of the media approaches.
0
R. Toby Richards
Asked:
R. Toby Richards
  • 3
  • 3
  • 2
  • +1
1 Solution
 
bbaoIT ConsultantCommented:
i think you need a complete Content Management System (CMS) to manage all digitalised data, not only because of its huge amount of data, but also for a proper, efficient way to access, back up and update the historic data.

once a right CMS is selected, you may design its storage back-end accordingly.
0
 
R. Toby RichardsNetwork AdministratorAuthor Commented:
The CMS is being provided by the vendor.
0
 
bbaoIT ConsultantCommented:
you mean the storage vendor or provider?
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
R. Toby RichardsNetwork AdministratorAuthor Commented:
The vendor who is scanning the microfiche is providing a software content management system, but no hardware or backup solution.
0
 
Gerald ConnollyCommented:
An obvious starting point is LTO tape, it's got easy portability, robust and longevity
0
 
R. Toby RichardsNetwork AdministratorAuthor Commented:
Since PDF's are already compressed that would require about 40 tapes, which doesn't seem viable.
0
 
bbaoIT ConsultantCommented:
> The vendor who is scanning themicrofiche is providing a software content management system

did they provide you the hardware spec or requirements of the storage which is compatible to their software? e.g. RAID, NAS, SAN, or specific tape systems?
0
 
Handy HolderSaggar makers bottom knockerCommented:
HP E7W30A plus Smart Array P822 would do for about 120TB usable with 35 * 4TB nearline 7.2K SAS (about $40k), but as Gerald Connolly says you still need an LTO library to back it up onto and a midrange server to act as fileserver.
0
 
Gerald ConnollyCommented:
100TB is a lot of data, and yes it might be 40+ tapes, but you can put them on the shelf and if they are treated properly you will still be able to read them in 10+ years time (compatible hardware permitting). They are designed for this.

You can't do that with disk they just are not designed to sit on the shelf and preserve the data, and you would be looking at a similar number of disks anyway.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

  • 3
  • 3
  • 2
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now