• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 474
  • Last Modified:

Archiving.

Hi.

We (me and my work colleagues) looking for a solution.

We currently have around 600GB of "data" - (faxes, ultrasonic scans, shearography scans, invoices, credit notes, etc.). Well over 3 million files.

The data covers only a few years and we expect to have at least 200GB a year for the current formats.

We have been manually archiving to DVD, but this is a very cumbersome way of working as we need to constantly check when a backup is required and to not backup previously restored files.

The capacity is supplied by a 4TB SAN drive (which also contains our VMs for our servers as well as the SQL server databases).

We have found that whilst we are growing at least 200GB a year, we can see that for upto 1 month after creation a file has a high frequency of use. After about 6 months it has a very low level of use.

For files older than 1 year, it is a handful of files which are used and for a short amount of time.


In our minds we would like a system to be able to store the files on some sort of permanent disk (DVD though Blue-Ray is the newbie here and has a much higher capacity).

The system would show that the file is available (even though it now exists on a DVD and not the hard disk). If the file content is required, it is retrieved from the DVD automatically.

These files are written to frequently within the first month and hardly ever after 6 months.

Versioning of the files would be useful too. A file is restored, amended and eventually placed onto an archive. If, at some stage later, another restore is required, then the amended file would be the default to be restored, but the possibility to get to the original file would be useful.


I think increasing the capacity is a non starter as we are going to be introducing new formats into this setup and we think that this could increase our yearly capacity to around 350GB.

What systems do you know about and have used? I'm looking for any sort of recommendation. Please don't give me links to HP/IBM/etc. We have some idea of what is available, but are looking for experience/recommendations.

Regards,

Richard Quadling.
0
Richard Quadling
Asked:
Richard Quadling
4 Solutions
 
techzterCommented:
We use a digital archiving software in our office by Treeno systems..

http://www.treenosoftware.com/

The system has been solid. It allows for version history as you had mentioned, along with putting workflows into place.

It can store multiple file types as well as allowing you to search documents by OCR if that would be helpful.

I can try to help with specific questions about it if you have some. The support has been terrific from this company as well.

It is basically like an unlimited virtual filing cabinet that you can store information into. The information can also be duplicated out into a tape, or DVD for offsite storage.

The software is extremely customizable if you have a programmer with some .php experience. The manufacturer also runs classes to help in training those individuals as well.
0
 
Richard QuadlingSenior Software DeverloperAuthor Commented:
PHP? (http://www.experts-exchange.com/experts.jsp?etIndex=3&mid=248814&showMessages=false#expertTools).

We are  looking for some sort of automatic archiver (including hardware). A jukebox system with capacity to hold hundreds of disks.
0
 
techzterCommented:
What we use for the hardware side of things is a Dell PowerEdge server with a Dell PowerVault MD3000 for the drives. The software itself can run on any hardware. Well at least Windows/Linux.

http://www.dell.com/content/products/productdetails.aspx/pvaul_md3000?c=us&cs=555&l=en&s=biz
0
Free recovery tool for Microsoft Active Directory

Veeam Explorer for Microsoft Active Directory provides fast and reliable object-level recovery for Active Directory from a single-pass, agentless backup or storage snapshot — without the need to restore an entire virtual machine or use third-party tools.

 
Richard QuadlingSenior Software DeverloperAuthor Commented:
Hmm. I see. Instead of removing anything, just keep it all online. Probably find harddrives (and even their enclosures) are cheaper than junk boxes.

I'll pass this on to TPTB.
0
 
Mesik2ppCommented:
We are using for online store VM and for archive EMC Centera. Which is built for archiving

http://www.emc.com/products/family/emc-centera-family.htm
0
 
raj27962Commented:
We have a cheap sata shelf on our san we use for heirarchical storage, data that is not accessed for over 6 months is auto moved to the storage on the sata disk, this is done automatically since we use SLES which has inbuilt funtionality for heirarchical storage.

If any of the "archived" storage that is on the slower disks get requested it is moved tempoarily back to the faster storage, in effect all the data is allways available however costs are greatly reduced since as you know yourself FC disks/SCSI etc disks are more expensive than sata disk.
0
 
Richard QuadlingSenior Software DeverloperAuthor Commented:
TPTB have decided to simply add more storage.

It is out of my hands.
0

Featured Post

Veeam Disaster Recovery in Microsoft Azure

Veeam PN for Microsoft Azure is a FREE solution designed to simplify and automate the setup of a DR site in Microsoft Azure using lightweight software-defined networking. It reduces the complexity of VPN deployments and is designed for businesses of ALL sizes.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now