Solved

How to store, serve, and back up 100 TB of PDF files?

Posted on 2015-02-04
9
174 Views
Last Modified: 2015-02-06
My organization currently stores, serves, and backs up about 4 TB of data. Now, out of necessity, we are converting all of our microfilm to PDF files. A vendor is doing that part, and they estimate that once converted, there will be 100TB of PDF files. I don't even know where to begin with managing that much data.

The backup medium should have a long shelf life. Otherwise, we'll be constantly re-backing up the data as the expected shelf life of the media approaches.
0
Comment
Question by:R. Toby Richards
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
  • +1
9 Comments
 
LVL 37

Expert Comment

by:bbao
ID: 40590013
i think you need a complete Content Management System (CMS) to manage all digitalised data, not only because of its huge amount of data, but also for a proper, efficient way to access, back up and update the historic data.

once a right CMS is selected, you may design its storage back-end accordingly.
0
 
LVL 5

Author Comment

by:R. Toby Richards
ID: 40590031
The CMS is being provided by the vendor.
0
 
LVL 37

Expert Comment

by:bbao
ID: 40590055
you mean the storage vendor or provider?
0
Edgartown IT Case Study

Learn about Edgartown's quest to ensure the safety and security of the entire town's employee and citizen data. Read the case study!

 
LVL 5

Author Comment

by:R. Toby Richards
ID: 40590067
The vendor who is scanning the microfiche is providing a software content management system, but no hardware or backup solution.
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 40592412
An obvious starting point is LTO tape, it's got easy portability, robust and longevity
0
 
LVL 5

Author Comment

by:R. Toby Richards
ID: 40592508
Since PDF's are already compressed that would require about 40 tapes, which doesn't seem viable.
0
 
LVL 37

Expert Comment

by:bbao
ID: 40592919
> The vendor who is scanning themicrofiche is providing a software content management system

did they provide you the hardware spec or requirements of the storage which is compatible to their software? e.g. RAID, NAS, SAN, or specific tape systems?
0
 
LVL 55

Expert Comment

by:andyalder
ID: 40593935
HP E7W30A plus Smart Array P822 would do for about 120TB usable with 35 * 4TB nearline 7.2K SAS (about $40k), but as Gerald Connolly says you still need an LTO library to back it up onto and a midrange server to act as fileserver.
0
 
LVL 17

Accepted Solution

by:
Gerald Connolly earned 500 total points
ID: 40594886
100TB is a lot of data, and yes it might be 40+ tapes, but you can put them on the shelf and if they are treated properly you will still be able to read them in 10+ years time (compatible hardware permitting). They are designed for this.

You can't do that with disk they just are not designed to sit on the shelf and preserve the data, and you would be looking at a similar number of disks anyway.
0

Featured Post

Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Concerto Cloud Services, a provider of fully managed private, public and hybrid cloud solutions, announced today it was named to the 20 Coolest Cloud Infrastructure Vendors Of The 2017 Cloud  (http://www.concertocloud.com/about/in-the-news/2017/02/0…
When we purchase storage, we typically are advertised storage of 500GB, 1TB, 2TB and so on. However, when you actually install it into your computer, your 500GB HDD will actually show up as 465GB. Why? It has to do with the way people and computers…
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

739 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question