Data compression with Arcserve

Posted on 2006-03-24
Last Modified: 2012-05-05
I have a Sony SDX-500C tape drive with AIT2 tapes with a stated native capacity of 50Gb and a compressed capacity of 130Gb.
I am running Arcserve 2000 version 7.0 on a Windows 2000 server.
The amount of data to backup has recently risen above 50Gb and is prompting for a second backup tape.
When I set the backup job to compress the data, I get the message w3032 Compression is disabled due to hardware data compression in the activity log.
My question is, if the hardware compression is enabled, then why am I only getting the native capacity of the tape?
Are the tapes specific to software compression only and should I be disabling the hardware compression to be able to do this? What impact is this likely to have on backup job?
Question by:wsmyth
    LVL 30

    Accepted Solution

    You rarely get what the compressed capacity says because of the files that are compressed.
    Word doc and text files might get you 2.5 to 3X compression,but exe,DB,and files like jpg or mpeg don't compress at all.
    LVL 22

    Expert Comment

    Compression is working and there is no need to recofigure. What it comes down to is a very poor rate of compression.

    The 50gb native noncompressed capacity is set by the engineers. That is the amount they know the drive will always right to tape unless there is a problem.

    The 130gb compressed capacity is set by the marketing heads. That is the amount they decided to publish as a capacity when compression is used.

    Point in fact the compressed capacity could be 51gb or it could be 500gb all depending on the type of data used.

    For example the ARCserve Tape Engine has a debug log. I have a 9gb tape log that compressed down to 33gb. On the other hand if you had 9gb of zip, mp3, jpg, or any other type of compressed data it would take a little more than 9gb on tape.

    What it comes down to is that it is not possible to compress compressed data, and those type of files are already compressed.  

    So there really is not anything you can do to make more data fit on tape. Provided of course there is not a problem, such as media or soft write errors. These type of errors will wast space on tape. Also backing up a lot of small files has more overhead than a few very large ones.
    LVL 2

    Expert Comment

    Also- keep in mind that hardware data compression is only effective if you can feed data to the tape drive fast enough, otherwise the algorithm will stream to tape at lower compression to keep the tape flowing and avoid back-hitching, also commonly referred to as "shoe-shining".  

    I find this to be the most common reason that tapes don't compress.  And since ArcServe has a terrible tape engine, it usually does not stream data as fast as other engines.  

    Back-hitching also often results in tape drive failure over time, more stress on the moving parts.

    LVL 9

    Expert Comment

    If you turn off hardware compression (there is a set off jumpers at the back of the casing to do this, I think it is jumper 7, but check the documentation) then you will be able to enable software compression.

    This is generally slower and not as efficient as hardware compression, but if you problem is shoe-shining (and it probably is if you have lots of very small files) then this will show it up. As the software is performing the compression on each file before sending it to the tape controller, you will definitely get your files compressed.

    All this being said, the fact still remains that advertising lies and you would very rarely get the advertised rates. Also, if you have small files, and mostly pre-compressed files (such as ZIP, JPG, MP3 etc) then you won't get any more compression.

    Hope this helps
    IPKON Networks Ltd

    Author Comment

    Prior to reading these answers I had read arcserves documentation on hardware and software compression. I had to completely wipe a tape to remove hardware compression settings. I could then disable hardware compression and set the software compression in the backup job. This appears to have been more successful than the hardware compression as the backup completed okay.
    My interpretation of what you are saying and would make sense, is that the marketing and labeling of these tapes are incorrect or deliberatly misleading to say the least. The physical tape can only store a maximum of 50Gb of data. If that data has been compressed from 130Gb down to 50Gb then the tape can store it. If the 130Gb cannot be compressed down to 50Gb, then it cannot be successfully backed up onto one tape.
    This is were the marketing and labeling is misleading because it gives the impression that the tape is capable of storing 130Gb of data provided that data is compressed to 130Gb. The physical drive can only store a maximum of 50Gb of data whether that data be compressed or not.
    LVL 22

    Expert Comment

    Yes, that is it and put very well.
    LVL 87

    Expert Comment

    Also, a used tape will normally have the compression set from the previous usage, so whenever you use that tape again it'll use the compression that was set previously except if you first "wipe" the tape like you just did. A new tape would probably also have given you better compression.
    LVL 1

    Expert Comment

    Did you ever get any compression with your tapes, even if you try to backup  .txt or database files, which should get good compression rates?
    Arcserve doesn't show any compression rate from the AIT-drive/tape or doesn't get these information from the drive/tape.
    I was able to put 250GB on a AIT-3 tape (100GB nativ / 260 GBcompressed) the data was a 1.2GB database file, which I copied 10times, to get a job of 12GB. Then I run the job 20 times and appended the data on the same media. That test showed that compression does work with some data files.
    Another thing, is that the tape must be somehow <<prepared>> for compression.
    Have a look at this article

    Author Comment

    One of the tests I tried before changing the compression from hardware to software was to identify a number of folders containing data that amounted to less than 50Gb and therefore would not need compressed. The folders contained a mixture of data and zipped files. Hardware compression was enabled. I set arcserve to compress the backup job as well. The message was software compression was unable to take place because hardware compression was set. No compression of data took place at all. Perhaps in order for hardware compression to take place when enabled, the data must be greater than the native capability and must also be capable of compression down to that native capability. What I know is the backup job I had trouble with completing using hardware compression, gave me no trouble when changing to software compression. Perhaps a previous comment about arcserves tape engine is the problem, perhaps your link is on to something.
    LVL 1

    Expert Comment

    As far as I've seen there seems to be no communication between Arcserve and the Sony drive about the hardware compression ratio. Arcserve only detects that the hardware compression is enabled and won't use the software compression.
    When I did the test with the 12GB job, I only knew the compression was working when I passed the 100GB native capacity limit of the tape straight up to 250GB of data on the tape. In none of the log files I did see an compression ratio.
    In the arcserve database record of the used media, I had 20 sessions with 12GB each, so I knew the hardware compression is working.
    It's a pitty that Arcserve or the drive doesn't show any compression ratio!!!!!!!!! So you would know immediately what's going on!
    Until now, I've found no other article which tells that the AIT tapes should be <<prepared>> for compression, but it seems to work somehow?

    Featured Post

    How your wiki can always stay up-to-date

    Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
    - Increase transparency
    - Onboard new hires faster
    - Access from mobile/offline

    Join & Write a Comment

    Problem description :  Some external hard disks / USB flash drives do not show actual space as mentioned in the factory settings. This is a common problem when you use an 8 GB USB drive to make it bootable to install a firmware/ driver on a serv…
    Lets start to have a small explanation what is VAAI(vStorage API for Array Integration ) and what are the benefits using it. VAAI is an API framework in VMware that enable some Storage tasks. It first presented in ESXi 4.1, but only after 5.x sup…
    This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
    This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

    754 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    21 Experts available now in Live!

    Get 1:1 Help Now