DAT tapes data transfer question

You could do all the above.    Cheapest and easiest is probably external hard drives.  

As far as directly copying data from the DAT drives to a different medium -- no, that's not a choice unless the tapes were written by DATMAN.

Much depends on the format in which the DAT tapes were originally written.  If they were written by Windows Backup or any other backup program, they must first be restored to a disk drive.  If they were written by something else, they must be processed using the program that wrote them.

Given that terabyte drives are below $100 and that a DDS-2 DAT tape holds on the order of 4 GB, I would restore them to two disk drives, then put one drive on the shelf for a safe backup copy.  Copying a 4 GB DDS-2 DAT to a 64 GB flash drive (more expensive than a 1500 GB disk drive) is a loss in cost/GB compared to disk; copying it to a 4 GB DVD-ROM is not an improvement in space, a breakeven in physical size, and only a marginal improvement in access speed, given that it's still a loadable medium that has to be located and loaded.

I received two responses, that say the opposite things.  One says:

GeneralTackett:
You could do all the above.    Cheapest and easiest is probably external hard drives.  

The other says:

DrKlahn:
As far as directly copying data from the DAT drives to a different medium -- no, that's not a choice unless the tapes were written by DATMAN.

Much depends on the format in which the DAT tapes were originally written.  If they were written by Windows Backup or any other backup program, they must first be restored to a disk drive.  If they were written by something else, they must be processed using the program that wrote them.

Given that terabyte drives are below $100 and that a DDS-2 DAT tape holds on the order of 4 GB, I would restore them to two disk drives, then put one drive on the shelf for a safe backup copy.  Copying a 4 GB DDS-2 DAT to a 64 GB flash drive (more expensive than a 1500 GB disk drive) is a loss in cost/GB compared to disk; copying it to a 4 GB DVD-ROM is not an improvement in space, a breakeven in physical size, and only a marginal improvement in access speed, given that it's still a loadable medium that has to be located and loaded.

The question is posed because I have about 40 DAT tapes of which I need a copy, because the tapes will be transferred to another owner.  So, I'm looking or the most economical way to get the data copied/transferred.

I'm not exactly sure how the tapes were created (done prior to my time), however most came from older HP servers that have loadable media slots for the tapes

I wondered if I could reload the tapes and copy the data or will I have to employ a third party vendor to achieve the copies?  Is there equipment I can purchase to do this myself?  If so what is it called and where could I get it?

Thanks for your help.

I'm not aware of any generic system that can directly transfer DAT tapes to removable media - as far as i know you have the issue that the backup system which is use changes the structure of how the data is stored... if i were in your situation i'd be looking at a restoration on HD's and transfer to removable media using some DVD/Blueray writing software.

Alternativly if you are looking at a solution that is scalable going forward then maybe use some backup software which offers intermediate backups i.e disk -> 2nd tier storage (disks) -> DAT. Based on this you always have the 2nd tier storage that you can keep in a safe/data storage facility.

The thing is, with current prices storage on disks tends to be as economical, or sometimes more economical that DAT.

Assuming you have 40 x 4GB tapes, then get a pair of the smallest disks you need and give them a pair with the same data on it....

I'm not exactly sure how the tapes were created (done prior to my time), however most came from older HP servers that have loadable media slots for the tapes

Oh dear.

The first thing you must do is find out how the tapes were created.  What software was used, and what revision level of the software.  Backup programs are not generally interchangeable in their backup formats.

Tape is not like disk; it doesn't contain files per se.  It contains data blocks larger than the 512 byte sector size of a disk, because big blocks are more efficient on tape.  Those blocks contain data that, when read by the proper restore program, can be used to regenerate the original files.

It's a little difficult to answer your question without some more information.

What size DAT tapes? (DDS1, DDS2...)

Are the HP servers that created these tapes still available and in working order? What OS do they run? What program created the tapes?

Backup tapes can have just about any type of data on them. There could be copies of files, vendor specific compressed data or a disk image. It's hard to tell...

In general you want to save data to magnetic media (hard drives, MO disks or tape) for long term storage. Most of the early CD-R disks are already unreadable. OTOH the first videotapes ever made (2" quad) still work.

I understand that I can put the data on external media.  Thanks everyone for clarifying that part.  However, I'm not exactly clear how to get the data from the tapes to the external media.  Can I load the DAT tapes (one at a time) back to the server they were backed up from and 'restore' the data (somehow) and then move the 'restored' data to the external media?  

ah I agree that just converting the tape.. isnt the idea.  but I presume you are using the same backup/restore software which typically will allow a restore to any mappable destination.

To load the data back you need to know which backup software was used, essentially different software has different structures and they wont interoperate (some will though). Maybe backup exec was use, maybe netbackup, maybe arcserve ... all are different and ideally you need to know which was used before you can restore.

Don't have the specifics on the tapes on hand right now.  The servers are still available and in working order.  They are running (sadly) Windows Server 2000.  I don't have details on what program created the tapes, I'm assuming the backup utility that came with the server.

external backup software was not likely used...

You might want to look at the scheduler on the W2000 machines. It could give a clue to what software was used.

"Your next step is to get on the server(s) that created the tapes and see if you can find a program to restore the data. If you are lucky you will be able to restore directly to a external USB hard drive. If not restore to a new empty directory on the W2000 server and then copy to a external USB hard drive."

Thanks for this, lewisg.

This was my next course of action (at least in my mind), but I was not sure if it would be possible (and still not sure, given what I don't know about the tapes).

Will it be as simple as reloading the tapes, "restoring" using (i'm hoping) the utility used to backup in the first place -- to a an external USB hard drive?  I would hope to be able to "swap and load" to get all of the data copied.  Am I thinking about this appropriately?

Alternatively, if I am unable to determine the original backup process am I at the mercy of a third party vendor?

Do you know approximately how long it would take to restore a single tape?

Thanks everyone for your recos.  I'll give the reload process a go and attempt to restore to an external drive.

No worries.

"There is a critical 'scrubbing' process that runs in powered disks"

Could you provide a cite for more information on this?

Thanks SelfGovern!

I have never seen this problem and I deal with a fair amount of older equipment that stays powered off for long periods of time. I do think the point of larger drives being a bigger problem is correct. I really liked this comment an article linked at the bottom of your link:

Last April at NAB, I felt like I was watching a number of camera manufacturers jump off the cliff of tapeless acquisition -- hoping that someone would invent an archive parachute before they smashed on the rocks below.

I'm still a big fan of tape. It is proven and lasts a long time. The current rush to disk based systems like RDX scares me. That tape is no longer supported in W7 is a problem. I work mostly with networks under 70 users and their data can still be stored on LTO1 drives.

In many ways the question here is what size and what sort of backup is on the DSS (DAT being the digital audio tape format DDS was derived from) tapes in question. If the tapes are just data and are DDS4 then LTRDEN may be looking at as much as 1.6T of data. More likely there is a lot of redundancy and possibly system files that are on the tapes. Careful pruning of this could reduce the disk space required quite a bit.

Another issue that has not been discussed is spanning of tapes. It is possible that some of the backups required more than one tape. In that case LTRDEN will have to figure out which tapes go with which set, hopefully they are marked...

Long term it is very important to keep your data sorted and refreshed. By getting rid of cruft on a regular basis you can be sure the good stuff is saved to currently accessible media.

Thanks SelfGovern for your added expertise!  Very helpful.

Again..thanks very much!  All of these responses have been greatly helpful.

You should probably close the thread so that we all dont have to monitor it anymore :)