Server freezes when backing up Outlook PST files using rsync
We are running a number of Windows XP PCs with MS Outlook (a mixture of Outlook XP and Outlook 2003) and a POP connection. The Outlook PST files are mostly stored on a Linux/qmail server so that they are backed up nightly with our other files. Backups are run using rsync to connect to a Clearlight portable hard drive using a USB connection. They are run in the middle of the night when local PCs are logged out or switched off.
This worked well until last December. Since then, the backup has had regular problems with PST files. On a frequent basis, but not nightly, the server crashes when backing up particular PST files. There are no error messages; the server response just grinds to a halt.
We have not been able to determine any pattern on when the crashes occur. It doesn’t always happen on the same file; there doesn't seem to be any link to file size or Outlook version.
We would really appreciate any ideas on how to solve this. At the moment we need to restart our server once or twice every week after a "backup crash".
This worked well until last December. Since then, the backup has had regular problems with PST files. On a frequent basis, but not nightly, the server crashes when backing up particular PST files. There are no error messages; the server response just grinds to a halt.
We have not been able to determine any pattern on when the crashes occur. It doesn’t always happen on the same file; there doesn't seem to be any link to file size or Outlook version.
We would really appreciate any ideas on how to solve this. At the moment we need to restart our server once or twice every week after a "backup crash".
rindi
Try changing the USB drive, or connect it directly to an internal port (IDE) on the server and then try running backups that way. If you can remove then disk from the USB case you should also use the disk manufacturer's utility to thoroughly test the drive, and maybe do a lowlevel format.
ASKER
Thanks for your prompt response.
We have already tried a number of existing and new Clearlight USB drives, including recent replacement of our old 60GB drives with newer 120GB drives.
We also have a backup onto an internal disk which works without problems. (I should have mentioned that earlier.) It is only the external backup which has the error.
We have already tried a number of existing and new Clearlight USB drives, including recent replacement of our old 60GB drives with newer 120GB drives.
We also have a backup onto an internal disk which works without problems. (I should have mentioned that earlier.) It is only the external backup which has the error.
Then it looks like a problem with your USB system. Have you tried other cables? Do the drives need separate power or do they draw all the power from the USB cable? If they take all the power directly from the USB, connect a powered USB hub between the server and the drives. If that doesn't help, get a PCI USB adapter and connect to that.
USB is a flakey, not ready for prime time production use technology.
If you want fast,hot swap disk backup,go eSATA.
.http://www.satasite.com/external-sata-esata-products.htm
If you want fast,hot swap disk backup,go eSATA.
.http://www.satasite.com/external-sata-esata-products.htm
I fully agree.
ASKER
Dear experts,
Thanks for your comments.
The USB drives are powered using a separate power pack. We have used different cables.
We have invested in Clearlight portable hard disks as they are reasonably robust to transfer daily to external storage and back. While suggestions of alternate hardware is appreciated, if possible, I would prefer not to replace all of our recently-purchased hardware immediately!
Thanks for your comments.
The USB drives are powered using a separate power pack. We have used different cables.
We have invested in Clearlight portable hard disks as they are reasonably robust to transfer daily to external storage and back. While suggestions of alternate hardware is appreciated, if possible, I would prefer not to replace all of our recently-purchased hardware immediately!
With all due respect,you should have researched using USB devices before making the commitment.of pruchasing those devices.
No production network should ever have to rely on USB devices for anything other than convience.
In my opinion ,having delt with USB for a couple of years,I can give insight as to it's lack of consistancy when used in this manner.
We all learn from our mistakes,so learn from this one.
The sooner you quit throwing time and effort at a non solution to your problem,the better off you will be.
If you do a search on this site for USB disk and problems ,you will see that there are a lot of issues with things like drives and data dissapearing for no known good reason.
Would you like to be in a posistion of trying to recover data from a backup and discover the entire disk is no longer recognized?
This happens far to often with USB.
Printers,thumb drives,keyborads and mice are OK for production USB,but I would never want to relay on it for critical data backups.
No production network should ever have to rely on USB devices for anything other than convience.
In my opinion ,having delt with USB for a couple of years,I can give insight as to it's lack of consistancy when used in this manner.
We all learn from our mistakes,so learn from this one.
The sooner you quit throwing time and effort at a non solution to your problem,the better off you will be.
If you do a search on this site for USB disk and problems ,you will see that there are a lot of issues with things like drives and data dissapearing for no known good reason.
Would you like to be in a posistion of trying to recover data from a backup and discover the entire disk is no longer recognized?
This happens far to often with USB.
Printers,thumb drives,keyborads and mice are OK for production USB,but I would never want to relay on it for critical data backups.
ASKER
Thanks, PGM554
Your comments are appreciated.
Based on your commments, would I be right in thinking that you would diagnose our problems as solely due to USB rather than any issue with Outlook PST files (or the backup software)?
I don't understand why the problem suddenly arose last December after working well for many months before then.
Your comments are appreciated.
Based on your commments, would I be right in thinking that you would diagnose our problems as solely due to USB rather than any issue with Outlook PST files (or the backup software)?
I don't understand why the problem suddenly arose last December after working well for many months before then.
Novell uses rsync in their iFolder product.
I have seen posts of PST files being not supported or issues arising when attempting the use of.
So, while I cannot completely rule out RSYNC, I would; look at USB as more of a culprit than the sync program
If Rsync works fine on the internal HD and not USB, the smoking gun appears to be USB
I have seen posts of PST files being not supported or issues arising when attempting the use of.
So, while I cannot completely rule out RSYNC, I would; look at USB as more of a culprit than the sync program
If Rsync works fine on the internal HD and not USB, the smoking gun appears to be USB
Remember, that a PST file is like any other DB file, if it is open, then issues may arise with it's ability to be backed up cleanly.
How big are the files that fail?
ASKER
They vary in size, say half to 1 GB. But other, larger files don't fail. And the same files don't fail consistently.
>>And the same files don't fail consistently.
This would make me think that you have usewrs who are logged in with Outlook open at the time of the backup. WithOutlook open the file will be locked and you won;t be able to back it up...
This would make me think that you have usewrs who are logged in with Outlook open at the time of the backup. WithOutlook open the file will be locked and you won;t be able to back it up...
ASKER
I wish it were that simple!
The backup has definitely crashed with users who were not only logged out but had turned off their machines. If a file were open, normally the backup would just skip it.
The backup has definitely crashed with users who were not only logged out but had turned off their machines. If a file were open, normally the backup would just skip it.
Rsync isn't ordinary backup up software ,it is syncronization,a whole different beast.
ASKER
Thanks for the clarification, pgm554. I didn't fully appreciate the distinction.
That said, I'm pretty sure that the crashes were not caused by PCs being left logged on overnight.
That said, I'm pretty sure that the crashes were not caused by PCs being left logged on overnight.
>>That said, I'm pretty sure that the crashes were not caused by PCs being left logged on overnight.
I don't know anything about Rsync having not used it, but a locked file is a locked file. So I googled "rsync pst" and the very first hit it returned was about an error with rsync and pst and Outlook open...
http://lists.samba.org/archive/rsync/2005-September/013582.html
Not sure how many users you're dealing with but; one day I would say we need to be down for some maintenance in the evening (or whatever works) and confirm everyone is off and see how your backup runs.
I don't know anything about Rsync having not used it, but a locked file is a locked file. So I googled "rsync pst" and the very first hit it returned was about an error with rsync and pst and Outlook open...
http://lists.samba.org/archive/rsync/2005-September/013582.html
Not sure how many users you're dealing with but; one day I would say we need to be down for some maintenance in the evening (or whatever works) and confirm everyone is off and see how your backup runs.
ASKER
Thanks, p912s
I had seen that post, but I think our problem is different.
Firstly, the PCs which use the PST files are switched off, so Outlook is definitely not open on those PCs when the backup runs in the middle of the night.
Secondly, although our server grinds to a halt (which was not mentioned in the post you quote), the PST files are not corrupted to the point of not opening. I used to run scanpst on whichever file crashed the server, but don't any more. Sometimes scanpst found problems; sometimes it didn't.
Also, most nights the backup works. But once or twice every week or two (sometimes more) the server needs to be rebooted after grinding to a halt during the backup.
thanks again for your comments.
I had seen that post, but I think our problem is different.
Firstly, the PCs which use the PST files are switched off, so Outlook is definitely not open on those PCs when the backup runs in the middle of the night.
Secondly, although our server grinds to a halt (which was not mentioned in the post you quote), the PST files are not corrupted to the point of not opening. I used to run scanpst on whichever file crashed the server, but don't any more. Sometimes scanpst found problems; sometimes it didn't.
Also, most nights the backup works. But once or twice every week or two (sometimes more) the server needs to be rebooted after grinding to a halt during the backup.
thanks again for your comments.
ASKER
We're currently working on another option involving copying PST files separately without using rsync and will post any results.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.