Backup Exec erratic compression ratios
Hi There,
Our shop is running nightly full backups of approx. 250GB on a 200/400GB LTO2 Drive using Backup Exec 11d on two Windows SBS 2003 machines connected over a network. The bulk of the data is on the server with the remote agent (I didn't configure it). Things generally run smoothly, except lately we have been experiencing tapes filling up due to the compression ratio going from roughly 1.47:1 down to about 1.15:1, and requesting a second tape. It doesn't happen regularly, and there's no obvious cause for the random drop in compression.
There have been errors on the backups where compression drops, but to me they don't seem related:
V-79-57344-33928 - Access is denied.
Access denied to directory data\(insert path here)\.
Directory \\SERVER\C:\data\(insert path here) was not found, or could not be accessed.
None of the files or subdirectories contained within will be backed up.
And on a separate occasion,
WARNING: "\\SERVER\C:\data\sys\eng mail\archive.pst" is a corrupt file.
This file cannot verify.
WARNING: "\\SERVER\C:\data\sys\eng mail\ARCHIVE_in.pst" is a corrupt file.
This file cannot verify.
WARNING: "\\SERVER\C:\data\sys\eng mail\ARCHIVE_in2.pst" is a corrupt file.
This file cannot verify.
WARNING: "\\SERVER\C:\data\sys\eng mail\ARCHIVE_SENT.pst" is a corrupt file.
This file cannot verify.
WARNING: "\\SERVER\C:\data\sys\eng mail\ARCHIVE_SENT2.pst" is a corrupt file.
This file cannot verify.
I've run into open .pst files before that would cause a "failure" but all the data still fit on the tape, and every other backup works fine with ~30GB free space to spare.
I updated Backup Exec with LiveUpdate, updated the SCSI drivers, but if things were working before I don't know how this would fix it. Also, no relevant looking errors in the event log. Any ideas what might be causing the compression ratio to go to hell randomly??? Thanks for all your help...
Our shop is running nightly full backups of approx. 250GB on a 200/400GB LTO2 Drive using Backup Exec 11d on two Windows SBS 2003 machines connected over a network. The bulk of the data is on the server with the remote agent (I didn't configure it). Things generally run smoothly, except lately we have been experiencing tapes filling up due to the compression ratio going from roughly 1.47:1 down to about 1.15:1, and requesting a second tape. It doesn't happen regularly, and there's no obvious cause for the random drop in compression.
There have been errors on the backups where compression drops, but to me they don't seem related:
V-79-57344-33928 - Access is denied.
Access denied to directory data\(insert path here)\.
Directory \\SERVER\C:\data\(insert path here) was not found, or could not be accessed.
None of the files or subdirectories contained within will be backed up.
And on a separate occasion,
WARNING: "\\SERVER\C:\data\sys\eng mail\archive.pst" is a corrupt file.
This file cannot verify.
WARNING: "\\SERVER\C:\data\sys\eng mail\ARCHIVE_in.pst" is a corrupt file.
This file cannot verify.
WARNING: "\\SERVER\C:\data\sys\eng mail\ARCHIVE_in2.pst" is a corrupt file.
This file cannot verify.
WARNING: "\\SERVER\C:\data\sys\eng mail\ARCHIVE_SENT.pst" is a corrupt file.
This file cannot verify.
WARNING: "\\SERVER\C:\data\sys\eng mail\ARCHIVE_SENT2.pst" is a corrupt file.
This file cannot verify.
I've run into open .pst files before that would cause a "failure" but all the data still fit on the tape, and every other backup works fine with ~30GB free space to spare.
I updated Backup Exec with LiveUpdate, updated the SCSI drivers, but if things were working before I don't know how this would fix it. Also, no relevant looking errors in the event log. Any ideas what might be causing the compression ratio to go to hell randomly??? Thanks for all your help...
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I tried the SGMON utility for Friday's backup, and there were numerous lines that read as follows:
[3472] HARDWARE COMPRESSION ===> Compression is configurable.
And almost as many that say:
[3472] HARDWARE COMPRESSION ===> Setting compression on.
The backup did fit on the tape this time as usual, so I'm completely at a loss as to what would cause the tape to fill randomly.
[3472] HARDWARE COMPRESSION ===> Compression is configurable.
And almost as many that say:
[3472] HARDWARE COMPRESSION ===> Setting compression on.
The backup did fit on the tape this time as usual, so I'm completely at a loss as to what would cause the tape to fill randomly.
ASKER
The problem has not repeated itself since I posted this question. Backups have behaved normally with no compression problems and relatively little data changed. over the past 3 weeks.
ASKER
Ok, the problem has finally repeated itself, and two backups in a row nonetheless. As I said before, we regularly get 250GB+ on these tapes with 30GB to spare, and on these occasions we get hit with 230GB and the tape is full. There is something that is randomly causing a folder or group of folders to compress inefficiently, but I can't determine what folders, nor what would be causing it! Would it be possible that the compression ratio would drop due to system uptime or windows updates that are installed and awaiting a system restart?
As a side note, I have even created a separate backup procedure to back up jpeg files to disk and keep them out of the tape backup selections (there are about 30GB of jpegs on the network drives) since they don't compress well. This has not seemed to help.
As a side note, I have even created a separate backup procedure to back up jpeg files to disk and keep them out of the tape backup selections (there are about 30GB of jpegs on the network drives) since they don't compress well. This has not seemed to help.
hi,
compression is something which has always been a point of contention between the hardware vendors and the backup software vendors.
One possible reason why you compression ratio is lower on one day and better on the other day can also be related to the media condition.
Check for tape errors in backup exec. check if there are any soft or hard errors in the log.
Normally when a backup runs if there are areas where tape has gone bad, it generally skips that areas which results in less amount of space available for data to be written, because of which it may span to new tape.
bhanu
compression is something which has always been a point of contention between the hardware vendors and the backup software vendors.
One possible reason why you compression ratio is lower on one day and better on the other day can also be related to the media condition.
Check for tape errors in backup exec. check if there are any soft or hard errors in the log.
Normally when a backup runs if there are areas where tape has gone bad, it generally skips that areas which results in less amount of space available for data to be written, because of which it may span to new tape.
bhanu
ASKER
Would the degraded media condition be reflected in the tape's total capacity under Media details? These tapes we're using are showing hundreds of thousands of soft-write errors, and very few other hard or soft errors (most others are soft read errors, and all tapes have <20). The tapes are about a year old and all have about 40 mounts in their histories.
I'm going to run a head cleaning (done regularly) and reboot and see if that makes any difference.
I'm going to run a head cleaning (done regularly) and reboot and see if that makes any difference.
yes the degraded media condition is what is reflected in the soft read and hard errors and when a next backup runs it does not write the data to those reported areas. Which in a sense leads to less amount of space available on the tapes.
ASKER
I'm still having a hard time understanding why this would affect the compression ratio. Normal backups in the past have given 1.5:1 (usually slightly better) but with the past 3 days' worth I'm only getting 1.14:1. Total capacity is still reported as 204GB on every tape... What am I missing?
hi,
when you say total capacity that is the amount of data written to the tape or free space on the tape. The compression is always compared with the available free space and space used for the backup.
bhanu
when you say total capacity that is the amount of data written to the tape or free space on the tape. The compression is always compared with the available free space and space used for the backup.
bhanu
ASKER
Right, the "total" capacity on every tape is showing as 204GB like it should... we have regularly put 250GB of data on those tapes. A typical backup for us looked like this:
Data: 244GB
Used capacity: 161GB
Available capacity: 42.9GB
Total Capacity: 204GB
Compression Ratio: 1.52:1
But the past few days has been this:
Data: 233GB
Used Capacity: 204GB
Available Capacity: 0 Bytes
Total Capacity: 204GB
Compression ratio: 1.14:1
I apologize for sounding redundant or if I'm completely missing something you're getting at. Somehow there's a huge chunk (50GB?) of data that is suddenly not compressing anywhere close to where it used to. Could this be a tape drive issue, a windows issue, a hard drive issue?
Data: 244GB
Used capacity: 161GB
Available capacity: 42.9GB
Total Capacity: 204GB
Compression Ratio: 1.52:1
But the past few days has been this:
Data: 233GB
Used Capacity: 204GB
Available Capacity: 0 Bytes
Total Capacity: 204GB
Compression ratio: 1.14:1
I apologize for sounding redundant or if I'm completely missing something you're getting at. Somehow there's a huge chunk (50GB?) of data that is suddenly not compressing anywhere close to where it used to. Could this be a tape drive issue, a windows issue, a hard drive issue?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
The bad sectors may be causing a decrease in capacity, but the tape is full regardless... it ejects and requests another tape to continue the backup.
As for the majority of the data that is being backed up:
Size: 250GB (268,582,810,629 bytes)
Size on Disk: 251GB (269,555,089,408 bytes)
From sifting through the data, there are few compressed files to speak of (i.e. zip) and all .jpg files are backed up by a different process; there are no compressed jpeg images in this backup set. Compression is not enabled on this drive.
As for the majority of the data that is being backed up:
Size: 250GB (268,582,810,629 bytes)
Size on Disk: 251GB (269,555,089,408 bytes)
From sifting through the data, there are few compressed files to speak of (i.e. zip) and all .jpg files are backed up by a different process; there are no compressed jpeg images in this backup set. Compression is not enabled on this drive.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I don't understand why one day the tape would fit 250GB (with 33GB still available on the tape) and the next day only fit 231GB (and the tape is full). The compression ratio dropped from 1.47:1 to 1.13:1 over the course of a day.
I will try the SGMON utility tonight to see if for some reason the compression is not behaving as it should though, thanks for the link.