Solved

DAT tapes data transfer question

Posted on 2010-11-16
29
948 Views
Last Modified: 2013-11-14
what is the quickest and most inexpensive way to transfer data from DAT tapes to a digital format (say DVDs or Large storage Flash Drives)?  Or can I reload the tapes and then copy the data to an external hard drive?
0
Comment
Question by:LTRDEN
  • 8
  • 8
  • 5
  • +5
29 Comments
 
LVL 8

Expert Comment

by:GeneralTackett
ID: 34152753
You could do all the above.    Cheapest and easiest is probably external hard drives.  
0
 
LVL 23

Expert Comment

by:Dr. Klahn
ID: 34152756
As far as directly copying data from the DAT drives to a different medium -- no, that's not a choice unless the tapes were written by DATMAN.

Much depends on the format in which the DAT tapes were originally written.  If they were written by Windows Backup or any other backup program, they must first be restored to a disk drive.  If they were written by something else, they must be processed using the program that wrote them.

Given that terabyte drives are below $100 and that a DDS-2 DAT tape holds on the order of 4 GB, I would restore them to two disk drives, then put one drive on the shelf for a safe backup copy.  Copying a 4 GB DDS-2 DAT to a 64 GB flash drive (more expensive than a 1500 GB disk drive) is a loss in cost/GB compared to disk; copying it to a 4 GB DVD-ROM is not an improvement in space, a breakeven in physical size, and only a marginal improvement in access speed, given that it's still a loadable medium that has to be located and loaded.
0
 

Author Comment

by:LTRDEN
ID: 34152790
I received two responses, that say the opposite things.  One says:

GeneralTackett:
You could do all the above.    Cheapest and easiest is probably external hard drives.  

The other says:

DrKlahn:
As far as directly copying data from the DAT drives to a different medium -- no, that's not a choice unless the tapes were written by DATMAN.

Much depends on the format in which the DAT tapes were originally written.  If they were written by Windows Backup or any other backup program, they must first be restored to a disk drive.  If they were written by something else, they must be processed using the program that wrote them.

Given that terabyte drives are below $100 and that a DDS-2 DAT tape holds on the order of 4 GB, I would restore them to two disk drives, then put one drive on the shelf for a safe backup copy.  Copying a 4 GB DDS-2 DAT to a 64 GB flash drive (more expensive than a 1500 GB disk drive) is a loss in cost/GB compared to disk; copying it to a 4 GB DVD-ROM is not an improvement in space, a breakeven in physical size, and only a marginal improvement in access speed, given that it's still a loadable medium that has to be located and loaded.

The question is posed because I have about 40 DAT tapes of which I need a copy, because the tapes will be transferred to another owner.  So, I'm looking or the most economical way to get the data copied/transferred.

I'm not exactly sure how the tapes were created (done prior to my time), however most came from older HP servers that have loadable media slots for the tapes

I wondered if I could reload the tapes and copy the data or will I have to employ a third party vendor to achieve the copies?  Is there equipment I can purchase to do this myself?  If so what is it called and where could I get it?

Thanks for your help.
0
 
LVL 20

Expert Comment

by:woolnoir
ID: 34152833
I'm not aware of any generic system that can directly transfer DAT tapes to removable media - as far as i know you have the issue that the backup system which is use changes the structure of how the data is stored... if i were in your situation i'd be looking at a restoration on HD's and transfer to removable media using some DVD/Blueray writing software.

Alternativly if you are looking at a solution that is scalable going forward then maybe use some backup software which offers intermediate backups i.e disk -> 2nd tier storage (disks) -> DAT. Based on this you always have the 2nd tier storage that you can keep in a safe/data storage facility.

The thing is, with current prices storage on disks tends to be as economical, or sometimes more economical that DAT.
0
 
LVL 20

Expert Comment

by:woolnoir
ID: 34152838
Assuming you have 40 x 4GB tapes, then get a pair of the smallest disks you need and give them a pair with the same data on it....
0
 
LVL 23

Expert Comment

by:Dr. Klahn
ID: 34152849
I'm not exactly sure how the tapes were created (done prior to my time), however most came from older HP servers that have loadable media slots for the tapes

Oh dear.

The first thing you must do is find out how the tapes were created.  What software was used, and what revision level of the software.  Backup programs are not generally interchangeable in their backup formats.

Tape is not like disk; it doesn't contain files per se.  It contains data blocks larger than the 512 byte sector size of a disk, because big blocks are more efficient on tape.  Those blocks contain data that, when read by the proper restore program, can be used to regenerate the original files.
0
 
LVL 7

Expert Comment

by:lewisg
ID: 34152854
It's a little difficult to answer your question without some more information.

What size DAT tapes? (DDS1, DDS2...)

Are the HP servers that created these tapes still available and in working order? What OS do they run? What program created the tapes?

Backup tapes can have just about any type of data on them. There could be copies of files, vendor specific compressed data or a disk image. It's hard to tell...

In general you want to save data to magnetic media (hard drives, MO disks or tape) for long term storage. Most of the early CD-R disks are already unreadable. OTOH the first videotapes ever made (2" quad) still work.
0
 

Author Comment

by:LTRDEN
ID: 34152861
I understand that I can put the data on external media.  Thanks everyone for clarifying that part.  However, I'm not exactly clear how to get the data from the tapes to the external media.  Can I load the DAT tapes (one at a time) back to the server they were backed up from and 'restore' the data (somehow) and then move the 'restored' data to the external media?  
0
 
LVL 8

Expert Comment

by:GeneralTackett
ID: 34152862
ah I agree that just converting the tape.. isnt the idea.  but I presume you are using the same backup/restore software which typically will allow a restore to any mappable destination.
0
 
LVL 20

Expert Comment

by:woolnoir
ID: 34152866
To load the data back you need to know which backup software was used, essentially different software has different structures and they wont interoperate (some will though). Maybe backup exec was use, maybe netbackup, maybe arcserve ... all are different and ideally you need to know which was used before you can restore.
0
 

Author Comment

by:LTRDEN
ID: 34152874
Don't have the specifics on the tapes on hand right now.  The servers are still available and in working order.  They are running (sadly) Windows Server 2000.  I don't have details on what program created the tapes, I'm assuming the backup utility that came with the server.
0
 

Author Comment

by:LTRDEN
ID: 34152880
external backup software was not likely used...
0
 
LVL 7

Accepted Solution

by:
lewisg earned 63 total points
ID: 34152897
Your next step is to get on the server(s) that created the tapes and see if you can find a program to restore the data. If you are lucky you will be able to restore directly to a external USB hard drive. If not restore to a new empty directory on the W2000 server and then copy to a external USB hard drive.

Be VERY careful that you don't overwrite any files on a working server, that could be VERY bad...
0
 
LVL 7

Expert Comment

by:lewisg
ID: 34152902
You might want to look at the scheduler on the W2000 machines. It could give a clue to what software was used.
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 
LVL 5

Assisted Solution

by:dacasey
dacasey earned 63 total points
ID: 34152916
Go buy a USB drive dock ($30) and a couple of SATA drives, restore directly to the disks.  Cheap easy and I can't think of anything faster.
0
 

Author Comment

by:LTRDEN
ID: 34152920

"Your next step is to get on the server(s) that created the tapes and see if you can find a program to restore the data. If you are lucky you will be able to restore directly to a external USB hard drive. If not restore to a new empty directory on the W2000 server and then copy to a external USB hard drive."

Thanks for this, lewisg.

This was my next course of action (at least in my mind), but I was not sure if it would be possible (and still not sure, given what I don't know about the tapes).

Will it be as simple as reloading the tapes, "restoring" using (i'm hoping) the utility used to backup in the first place -- to a an external USB hard drive?  I would hope to be able to "swap and load" to get all of the data copied.  Am I thinking about this appropriately?

Alternatively, if I am unable to determine the original backup process am I at the mercy of a third party vendor?

Do you know approximately how long it would take to restore a single tape?
0
 
LVL 20

Assisted Solution

by:woolnoir
woolnoir earned 188 total points
ID: 34152934
DDS-2 drives offer 4GB native capacity with up to 510KB per second  operation based on this it would take about 2.5 hours to restore the data if everything were optimal.
0
 
LVL 20

Assisted Solution

by:woolnoir
woolnoir earned 188 total points
ID: 34152941
>Alternatively, if I am unable to determine the original backup process am I at the mercy of a third party vendor?

Pretty much unfortunately - thats why DR strategies always have documentation and software media included as part of them, you are lucky you still have a DAT drive - many companies i've dealt with in the past moved from DAT to DLT drives and dumped their old equipment. WHen it came to doing a restore they have no drives to do it on, and a mountain of tapes.

0
 
LVL 20

Assisted Solution

by:woolnoir
woolnoir earned 188 total points
ID: 34152946
> Will it be as simple as reloading the tapes, "restoring" using (i'm hoping) the utility used to backup in the first place -- to a an external USB hard drive?  I would hope to be able to "swap and load" to get all of the data copied.  Am I thinking about this appropriately?

assuming the server has the backup software on it, you need to put the tapes in and generally there wil be a 'inventory' function or a 'import' function to scan the tapes (to determine whats on them). Do this, and then do a restore from media and it should ask for each tape in turn. This again depends on the software.
0
 

Author Comment

by:LTRDEN
ID: 34152959
Thanks everyone for your recos.  I'll give the reload process a go and attempt to restore to an external drive.
0
 
LVL 20

Expert Comment

by:woolnoir
ID: 34152990
No worries.
0
 
LVL 20

Assisted Solution

by:SelfGovern
SelfGovern earned 124 total points
ID: 34154654
A suggestion on restore methodology --

If it were me, I'd get a single enterprise-class 500GB external disk as decasey recomended.
For each tape, I'd create a directory on that disk that identified the tape in question -- that is, maybe the directory is the date the tape was made (in YYYY MM DD format, please!), or the tape ID, or something similar.
Then I'd restore that tape to that directory.  Repeat 39 times.  Depending on how much data is on each tape, you'll get a few to maybe 1/4 of your set done per day.

In the properties for the disk, mark it read-only -- either GUI or command line:
             attrib      +R     x:\*.*     /s
Substitute your drive letter for the 'x'.


Once that's done -- I'd make copies directory by directory from the hard drive to archival quality DVD +R media.  Make two copies if the data is important.  Archival quality media is *very* important for this, even if you only need a year or two retention.

DO NOT rely on a disk sitting on the shelf to be readable in a year or two down the road.  There is a critical 'scrubbing' process that runs in powered disks, ensuring that all sectors can be read, and rewriting them if the signal strength starts to drop... this doesn't happen on unpowered disks, and you may be losing data without knowing until it's too late.
0
 
LVL 7

Expert Comment

by:lewisg
ID: 34154992
"There is a critical 'scrubbing' process that runs in powered disks"

Could you provide a cite for more information on this?
0
 
LVL 20

Assisted Solution

by:SelfGovern
SelfGovern earned 124 total points
ID: 34157043
Lewisg, "scrubbing" was probably not the best word.   But please see the article at
                        http://www.larryjordan.biz/articles/lj_hard_disk_warning.html

which confirms that
- There is an automatic process that refreshes the data on your hard drive while powered on
- This is an essential process, and without it, the signals will degrade and data will be lost
- The length of data retention on unpowered hard drives is estimated at 12-18 months.

I suspect that the problem will be more severe with more dense drives (i.e., given the same
number and size of platters, a 2TB drive will probably be more susceptible to data loss than
a 1TB drive), with consumer- vs. enterprise-class drives, and with SATA vs. SAS drives.
0
 
LVL 7

Expert Comment

by:lewisg
ID: 34157438
Thanks SelfGovern!

I have never seen this problem and I deal with a fair amount of older equipment that stays powered off for long periods of time. I do think the point of larger drives being a bigger problem is correct. I really liked this comment an article linked at the bottom of your link:

Last April at NAB, I felt like I was watching a number of camera manufacturers jump off the cliff of tapeless acquisition -- hoping that someone would invent an archive parachute before they smashed on the rocks below.

I'm still a big fan of tape. It is proven and lasts a long time. The current rush to disk based systems like RDX scares me. That tape is no longer supported in W7 is a problem. I work mostly with networks under 70 users and their data can still be stored on LTO1 drives.

In many ways the question here is what size and what sort of backup is on the DSS (DAT being the digital audio tape format DDS was derived from) tapes in question. If the tapes are just data and are DDS4 then LTRDEN may be looking at as much as 1.6T of data. More likely there is a lot of redundancy and possibly system files that are on the tapes. Careful pruning of this could reduce the disk space required quite a bit.

Another issue that has not been discussed is spanning of tapes. It is possible that some of the backups required more than one tape. In that case LTRDEN will have to figure out which tapes go with which set, hopefully they are marked...

Long term it is very important to keep your data sorted and refreshed. By getting rid of cruft on a regular basis you can be sure the good stuff is saved to currently accessible media.
0
 

Author Comment

by:LTRDEN
ID: 34158209
Thanks SelfGovern for your added expertise!  Very helpful.
0
 

Assisted Solution

by:TristanB
TristanB earned 62 total points
ID: 34158760
LRTDEN,

Windows 2000 has an internal backup application called NTBackup. There is a very good chance that it was used to run the backup to the internal DAT drives if there isn't a 3rd party backup app installed.

Put a DAT tape into the internal Dat drive. Then  type 'ntbackup' from the run box and it will start the backup tool. It isn't too complicated to figure out and you will probably need to 'catalog' the tapes so that you can see the files in the backups. Once you have the catalog, you can browse the backup. Check the folders you want to restore.

Plug in a USB external drive, check the drive letter created and restore the data from tape to the USB drive.

0
 

Author Comment

by:LTRDEN
ID: 34161807
Again..thanks very much!  All of these responses have been greatly helpful.
0
 
LVL 20

Expert Comment

by:woolnoir
ID: 34264926
You should probably close the thread so that we all dont have to monitor it anymore :)
0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

Having issues meeting security compliance criteria because of those pesky USB drives? Then I can help you! This article will explain how to disable USB Mass Storage devices in Windows Server 2008 R2.
How to fix error ""Failed to validate the vCentre certificate. Either install or verify the certificate by using the vSphere Data Protection Configuration utility" when you are trying to connect to VDP instance from Vcenter.
This tutorial will walk an individual through the process of configuring basic necessities in order to use the 2010 version of Data Protection Manager. These include storage, agents, and protection jobs. Launch Data Protection Manager from the deskt…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now