dpm serious issues
Hi,
Are end of month backup to tape job started last week and on pretty much all 9 DPM Servers I am seeing the following error:
"This operation failed because of a protection agent faulire. (ID 998 Details. The device is not connected (0X8007048F) - Retry the Operation"
When looking into event viewer I can see the following entry:
The back up to tape job failed for the following reason: (ID: 3311)
The operation failed because of a protection agent failure. (ID: 998)
Backup job for datasource: Z:\ on production server: FILE.domain.internal failed.
Backup job failed at: 31/12/2015 02:51:53.
Backup Type: tape backup.
there is a entry for the tape library:
hplto: The device, \Device\TapeDrive0, is not ready for access yet.
I've spent a number of hours on this and getting no closer.............I'm pretty stumped due to this happening on MOST of are DPM Servers.........nothing has changed, monthlys have been running perfectly fine all year!
I've updated to the latest firmware and drivers for the tape drive and library
Are end of month backup to tape job started last week and on pretty much all 9 DPM Servers I am seeing the following error:
"This operation failed because of a protection agent faulire. (ID 998 Details. The device is not connected (0X8007048F) - Retry the Operation"
When looking into event viewer I can see the following entry:
The back up to tape job failed for the following reason: (ID: 3311)
The operation failed because of a protection agent failure. (ID: 998)
Backup job for datasource: Z:\ on production server: FILE.domain.internal failed.
Backup job failed at: 31/12/2015 02:51:53.
Backup Type: tape backup.
there is a entry for the tape library:
hplto: The device, \Device\TapeDrive0, is not ready for access yet.
I've spent a number of hours on this and getting no closer.............I'm pretty stumped due to this happening on MOST of are DPM Servers.........nothing has changed, monthlys have been running perfectly fine all year!
I've updated to the latest firmware and drivers for the tape drive and library
How do the DPM servers access the tape device? IP? Is the tape device functioning properly?
The error message, simply put, is saying that the server(s) cannot communicate with the tape device.
I would verify that the device is online and functioning properly.
Dan
The error message, simply put, is saying that the server(s) cannot communicate with the tape device.
I would verify that the device is online and functioning properly.
Dan
ASKER
Hi Dan,
Each DPM Server has there own tape library (MSL2024) and each server is connected via SAS.
Tape Libraries are online, I've checked the web GUI and no reported issues........
Each DPM Server has there own tape library (MSL2024) and each server is connected via SAS.
Tape Libraries are online, I've checked the web GUI and no reported issues........
Can you verify that the server hardware and the tape device are all running the latest BIOS/firmware and drivers?
Its an issue that has been seen before.
link: http://www.networksteve.com/enterprise/topic.php/Protection_Agent_failure_error_during_backup_to_tape_with_DPM_20/?TopicId=75676&Posts=0
Dan
Its an issue that has been seen before.
link: http://www.networksteve.com/enterprise/topic.php/Protection_Agent_failure_error_during_backup_to_tape_with_DPM_20/?TopicId=75676&Posts=0
Dan
ASKER
Thanks but I've already come across that thread, all tape libraries have the latest firmware and drivers.
Tape libraries have been rebooted, dpm servers rebooted.............
The strange thing is its all started at the same time even though everything has been working fine all year
Tape libraries have been rebooted, dpm servers rebooted.............
The strange thing is its all started at the same time even though everything has been working fine all year
Any new windows update that replaced the hp tape driver? Change in the driver to the HBA or raid card the tape is attached to? I've seen that in the Symantec world. The latest doesn't mean it necessarily works.
ASKER
None - Windows Updates is controlled by SCCM and we havent deployed any updated recently.
I'm stumped!
I'm stumped!
Any updates to DPM? Sccm service pack or anything?
ASKER
None, is it worth opening a case with Microsoft on this?
As its the holiday period no system changes have been made
As its the holiday period no system changes have been made
ASKER
Checked again, backup to tape jobs failing all over the place with the same error
ASKER
Just to update all on this, I have done the following:
1. Updated tape library to latest firmware and drives
2. Updated DPM 2012 R2 to latest Update Rollup
3. Latest Windows Update (for 2012 R2)
4. Reboot DPM Server
5. Rebooted Tape Library
6. Cleared DPM error logs
7. Done this: http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=mmr_kc-0100617
8. Check Server, Tape GUI for any hardware issues, none
9. HP ProLiant Server is fully updated with latest ProLiant Pack
10. Deployed latest DPM Agent to servers
11. No reported issues via PRTG, no drops on any servers or tape libraries
After all this the problem still remains
1. Updated tape library to latest firmware and drives
2. Updated DPM 2012 R2 to latest Update Rollup
3. Latest Windows Update (for 2012 R2)
4. Reboot DPM Server
5. Rebooted Tape Library
6. Cleared DPM error logs
7. Done this: http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=mmr_kc-0100617
8. Check Server, Tape GUI for any hardware issues, none
9. HP ProLiant Server is fully updated with latest ProLiant Pack
10. Deployed latest DPM Agent to servers
11. No reported issues via PRTG, no drops on any servers or tape libraries
After all this the problem still remains
Have you tried completely deinstalling the device and its drivers? Verifying that the device is no present for any of the local components (DPM, etc.), disconnecting the tape library, bouncing the server and doing a re-installation of the hardware?
Dan
Dan
This article may be worth a read. Mentions running a query to reset the info DPM stores on the tape libraries.
Link: http://scug.be/scdpm/2009/11/25/system-center-data-protection-manager-2007-tape-problems/
Dan
Link: http://scug.be/scdpm/2009/11/25/system-center-data-protection-manager-2007-tape-problems/
Dan
ASKER
Thanks Dan, will give it a read.
ASKER
Cant see the query.txt in that post! :(
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
it fixed my issue
ASKER
1184 206C 12/28 11:53:35.594 27 CommonErrorHandler.cs(173)