SQL 2000 Transactional Backup Failures.
Hey Guys,
After doing as much research as I can online I'm currently stuck. Plus my knowledge of sql 2000 is smaller then a grain of salt.
Just recently I have started to receive this error from our sql 2000 box.
JOB RUN: 'Transaction Log Backup Job for DB Maintenance Plan 'DataConnect'' was run on 1/3/2013 at 11:35:00 AM
DURATION: 0 hours, 28 minutes, 40 seconds
STATUS: Failed
MESSAGES: The job failed. The Job was invoked by Schedule 40 (Schedule 1). The last step to run was step 1 (Step 1).
I look at the history and the log file and this is what I find:
Message
Executed as user: SVAMAIN\sqlservice. sqlmaint.exe failed. [SQLSTATE 42000] (Error 22029). The step failed.
I then go to the SQL Server logs and view the current log.
At the same time when I received the email that the job failed I am able to find this:
Source: spid69
BackupDiskFile::RequestDur
When I go to the location of where the backup file is there are transactional logs there (whether they are complete or not is another question but data was written to the network locations).
Things have been going fine until today. Nothing on the network has changed. A few months back I had to move the backups from our SAN to our NAS due to storage issues, migrations, etc. The hop count hasn't changed and it's the same millsecond response time when doing a ping and tracert.
From all the research I can see is that SQL 2000 really doesn't like it when the network connection get's congested and slows down. Instead of continuing to keep running it fails the job. Is there a way to force the Maintenance plan to keep going regardless of network congestion or slow response times from the destination NIC?
Thank you for any help that you can provide :)
After doing as much research as I can online I'm currently stuck. Plus my knowledge of sql 2000 is smaller then a grain of salt.
Just recently I have started to receive this error from our sql 2000 box.
JOB RUN: 'Transaction Log Backup Job for DB Maintenance Plan 'DataConnect'' was run on 1/3/2013 at 11:35:00 AM
DURATION: 0 hours, 28 minutes, 40 seconds
STATUS: Failed
MESSAGES: The job failed. The Job was invoked by Schedule 40 (Schedule 1). The last step to run was step 1 (Step 1).
I look at the history and the log file and this is what I find:
Message
Executed as user: SVAMAIN\sqlservice. sqlmaint.exe failed. [SQLSTATE 42000] (Error 22029). The step failed.
I then go to the SQL Server logs and view the current log.
At the same time when I received the email that the job failed I am able to find this:
Source: spid69
BackupDiskFile::RequestDur
When I go to the location of where the backup file is there are transactional logs there (whether they are complete or not is another question but data was written to the network locations).
Things have been going fine until today. Nothing on the network has changed. A few months back I had to move the backups from our SAN to our NAS due to storage issues, migrations, etc. The hop count hasn't changed and it's the same millsecond response time when doing a ping and tracert.
From all the research I can see is that SQL 2000 really doesn't like it when the network connection get's congested and slows down. Instead of continuing to keep running it fails the job. Is there a way to force the Maintenance plan to keep going regardless of network congestion or slow response times from the destination NIC?
Thank you for any help that you can provide :)
are you using sql 2000 native backup? if 3rd party like the idera sqlsafe software there is an option write try for such network issues..
in any case check if your \\xxxxx server is busy that time: try to change bkp schedule or just set in job retry on failure , add more cpu on \\xxxxx ; faster hdd; nic , etc
in any case check if your \\xxxxx server is busy that time: try to change bkp schedule or just set in job retry on failure , add more cpu on \\xxxxx ; faster hdd; nic , etc
ASKER
Hey Kevin,
The NAS is connected via 1GB Ethernet. However there is a lot of other traffic happening with this NAS during all times of the day/night.
I did try the retry interval and the on failure action option. With the system trying more before failing i did get one to complete last night (probably on it's 2nd or 3rd try). However since these are also transactional backup I need them to complete during the day.
Hey EugeneZ,
I am using native SQL 2000. There is currently no 3rd party software being used. I've changed the schedule before with these backup's and they have helped but the issue continue's to pop up now and then again. Having to keep changing the schedule isn't a good enough option for a fix/solution for us.
The NAS may need a 2nd NIC installed which is what I am considering and need to find out if that's even possible.
The NAS is connected via 1GB Ethernet. However there is a lot of other traffic happening with this NAS during all times of the day/night.
I did try the retry interval and the on failure action option. With the system trying more before failing i did get one to complete last night (probably on it's 2nd or 3rd try). However since these are also transactional backup I need them to complete during the day.
Hey EugeneZ,
I am using native SQL 2000. There is currently no 3rd party software being used. I've changed the schedule before with these backup's and they have helped but the issue continue's to pop up now and then again. Having to keep changing the schedule isn't a good enough option for a fix/solution for us.
The NAS may need a 2nd NIC installed which is what I am considering and need to find out if that's even possible.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Currently still working on the 2nd NIC. I ran some perfmon logs and found out it's not an I/O issue. Memory and CPU usage is well below thresholds. Our Nas is a Dell PowerVault nx3100.
When I ran one full backup from our sql2000 server to the NAS I also along with the perfmon watch the NIC and saw that it was at a constant area of 20-25% usage.
Then another scheduled backup Ran about 45mins later and failed (another sql 2000 backup). At this point it still looks like a NIC over utilization/saturation issue?
I did look at the idera software and tried a trail backup. The backup ran nice and compressed to a really small size. This is something i may look at down the road.
When I ran one full backup from our sql2000 server to the NAS I also along with the perfmon watch the NIC and saw that it was at a constant area of 20-25% usage.
Then another scheduled backup Ran about 45mins later and failed (another sql 2000 backup). At this point it still looks like a NIC over utilization/saturation issue?
I did look at the idera software and tried a trail backup. The backup ran nice and compressed to a really small size. This is something i may look at down the road.
<20-25% usage.> check the NIC speed settings on the box. are these 2 teamed NICs? and btw are they 1Gbps or 100 Mbps? what is the router switch speed?
ASKER
There currently is only 1 NIC. I haven't teamed or done anything with a 2nd NIC yet.
The NIC ports are 1GB and the routers ports are 1GB as well.
The NIC ports are 1GB and the routers ports are 1GB as well.
ASKER
Thank you guys for your help both of you helped me answer the problem. It is with NIC/network issues and needing a 2nd NIC for traffic. I have obtained proof and can progress forward internally.
Anyway, if your question is how to get the job to continue on error, change the step's on failure procedure. If you expand SQL Server Agent > Jobs, right-click maintenance job, and select Properties, then you will be able to edit the individual Steps. Under the advanced options for each step, you will see "On failure action" which you can change to "Go to next step." This will ignore the error and continue process the job. You will also see options to set "Retry attempts" and the associated "Retry interval (minutes)."
I hope that helps!
Best regards and happy coding,
Kevin