Microsoft System Center Data Protection Manager won't open

bytespeed
bytespeed used Ask the Experts™
on
I am attempting to get our backups running again on Microsoft System Center Data Protection Manager 2010.   I didn't set this up so I am not overly familiar with how it works unfortunately.
The backup server stopped responding last week and after a reboot the backup software will no longer open.
When attempting to open it...It shows an error of "Microsoft System Center Data Protection Manager 2010 is unable to connect.....  ID 948.   Verify DPM service is running"
If I try to manually start the DPM service it flashes an error of "Could not start - error 1068"  The dependency group failed to start."
We are using SQL 2008.
I attempted to open the SQL 2008 database and when I tried to open it...it showed an error of:
"An exception occurred while executing a Transact-SQL statement or batch (Microsoft.sqlserver.connectioninfo)
Additional Info - Database MSDB cannot be opened.  It has been  marked as SUSPECT by recovery - error 926.
Any help would be greatly appreciated!
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Marten RuneSQL Expert/Infrastructure Architect

Commented:
MSDB cannot be opened, and your database is marked as SUSPECT!

I would guess you are on failing HW. Does your system reside om raid discs? Whats in the errorlog of SQL, Whats in Event viewer. What does a harddrive checkdisk say (run WO fixing anything, important)?

Time to check weather you have backups of the system and user databases?

Please provide this information and we will take it from there.

Regards Marten

Author

Commented:
Thanks for the reply.
The hardware right now seems good and hasn't acted up since it was rebooted last week.  It is running on a RAID volume and it is showing optimal status.
I ran a check disk and it stated that it found no problems.

Here is the part of the SQL error log when the server wasn't responding on 10/15 and a couple things from before it...
2010-10-13 23:30:02.33 spid19s        keylock hobtid=72057594043236352 dbid=7 objectname=DPMDB.dbo.tbl_TE_TaskTrail indexname=ix_TE_TaskTrail_VerbID_TaskID id=lock8d268700 mode=X associatedObjectId=72057594043236352
2010-10-13 23:30:02.33 spid19s         owner-list
2010-10-13 23:30:02.33 spid19s          owner id=process467ddc8 mode=X
2010-10-13 23:30:02.33 spid19s         waiter-list
2010-10-13 23:30:02.33 spid19s          waiter id=process3cf048 mode=S requestType=wait
2010-10-13 23:30:02.33 spid19s        pagelock fileid=1 pageid=25253 dbid=7 objectname=DPMDB.dbo.tbl_TE_TaskTrail id=lock858be600 mode=SIU associatedObjectId=72057594064404480
2010-10-13 23:30:02.33 spid19s         owner-list
2010-10-13 23:30:02.33 spid19s          owner id=process3cf048 mode=S
2010-10-13 23:30:02.33 spid19s         waiter-list
2010-10-13 23:30:02.33 spid19s          waiter id=process467ddc8 mode=IX requestType=convert
2010-10-14 00:00:42.51 spid18s     This instance of SQL Server has been using a process ID of 1984 since 9/30/2010 7:42:59 AM (local) 9/30/2010 12:42:59 PM (UTC). This is an informational message only; no user action is required.
2010-10-15 00:00:55.32 spid18s     This instance of SQL Server has been using a process ID of 1984 since 9/30/2010 7:42:59 AM (local) 9/30/2010 12:42:59 PM (UTC). This is an informational message only; no user action is required.
2010-10-15 08:17:11.87 Server      SQL Server is terminating because of a system shutdown. This is an informational message only. No user action is required.

---------------------------------------------------------
This should be from when it was powered back on again:

2010-10-15 11:33:31.17 Server      Authentication mode is WINDOWS-ONLY.
2010-10-15 11:33:31.17 Server      Logging SQL Server messages in file 'C:\Program Files\Microsoft DPM\SQL\MSSQL10.MSDPM2010\MSSQL\Log\ERRORLOG'.
2010-10-15 11:33:31.25 Server      This instance of SQL Server last reported using a process ID of 1908 at 10/15/2010 11:23:14 AM (local) 10/15/2010 4:23:14 PM (UTC). This is an informational message only; no user action is required.
2010-10-15 11:33:31.25 Server      Registry startup parameters:
       -d C:\Program Files\Microsoft DPM\SQL\MSSQL10.MSDPM2010\MSSQL\DATA\master.mdf
       -e C:\Program Files\Microsoft DPM\SQL\MSSQL10.MSDPM2010\MSSQL\Log\ERRORLOG
       -l C:\Program Files\Microsoft DPM\SQL\MSSQL10.MSDPM2010\MSSQL\DATA\mastlog.ldf
2010-10-15 11:33:31.47 Server      SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required.
2010-10-15 11:33:31.49 Server      Detected 4 CPUs. This is an informational message; no user action is required.
2010-10-15 11:33:33.10 Server      Using dynamic lock allocation.  Initial allocation of 2500 Lock blocks and 5000 Lock Owner blocks per node.  This is an informational message only.  No user action is required.
2010-10-15 11:33:35.03 Server      Node configuration: node 0: CPU mask: 0x000000000000000f Active CPU mask: 0x000000000000000f. This message provides a description of the NUMA configuration for this computer. This is an informational message only. No user action is required.
2010-10-15 11:33:37.45 spid6s      Starting up database 'master'.
2010-10-15 11:33:39.74 spid6s      Recovery is writing a checkpoint in database 'master' (1). This is an informational message only. No user action is required.
2010-10-15 11:33:39.97 spid6s      Recovery completed for database master (database ID 1) in 1 second(s) (analysis 362 ms, redo 357 ms, undo 355 ms.) This is an informational message only. No user action is required.
2010-10-15 11:33:42.57 spid6s      FILESTREAM: effective level = 0, configured level = 0, file system access share name = 'MSDPM2010'.
2010-10-15 11:33:44.70 spid6s      SQL Trace ID 1 was started by login "sa".
2010-10-15 11:33:45.19 spid6s      Starting up database 'mssqlsystemresource'.
2010-10-15 11:33:45.27 spid6s      The resource database build version is 10.00.2531. This is an informational message only. No user action is required.
2010-10-15 11:33:46.46 spid10s     Starting up database 'model'.
2010-10-15 11:33:46.49 spid6s      Server name is 'BACKUP\MSDPM2010'. This is an informational message only. No user action is required.
2010-10-15 11:33:46.75 spid6s      Informational: No full-text supported languages found.
2010-10-15 11:33:46.75 spid14s     Starting up database 'ReportServer$MSDPM2010TempDB'.
2010-10-15 11:33:46.75 spid13s     Starting up database 'msdb'.
2010-10-15 11:33:46.75 spid16s     Starting up database 'DPMDB'.
2010-10-15 11:33:46.78 spid15s     Starting up database 'ReportServer$MSDPM2010'.
2010-10-15 11:33:46.92 Server      A self-generated certificate was successfully loaded for encryption.
2010-10-15 11:33:46.95 Server      Server is listening on [ 'any' <ipv6> 51745].
2010-10-15 11:33:46.95 Server      Server is listening on [ 'any' <ipv4> 51745].
2010-10-15 11:33:46.95 Server      Server local connection provider is ready to accept connection on [ \\.\pipe\SQLLocal\MSDPM2010 ].
2010-10-15 11:33:46.95 Server      Server local connection provider is ready to accept connection on [ \\.\pipe\MSSQL$MSDPM2010\sql\query ].
2010-10-15 11:33:47.80 Logon       Error: 17187, Severity: 16, State: 1.
2010-10-15 11:33:47.80 Logon       SQL Server is not ready to accept new client connections. Wait a few minutes before trying again. If you have access to the error log, look for the informational message that indicates that SQL Server is ready before trying to connect again.  [CLIENT: 10.0.0.21]
2010-10-15 11:33:47.80 Logon       Error: 17187, Severity: 16, State: 1.
2010-10-15 11:33:47.80 Logon       SQL Server is not ready to accept new client connections. Wait a few minutes before trying again. If you have access to the error log, look for the informational message that indicates that SQL Server is ready before trying to connect again.  [CLIENT: 10.0.0.21]
2010-10-15 11:33:47.82 Logon       Error: 17187, Severity: 16, State: 1.
2010-10-15 11:33:47.82 Logon       SQL Server is not ready to accept new client connections. Wait a few minutes before trying again. If you have access to the error log, look for the informational message that indicates that SQL Server is ready before trying to connect again.  [CLIENT: 10.0.0.21]
2010-10-15 11:33:47.82 Logon       Error: 17187, Severity: 16, State: 1.
2010-10-15 11:33:47.82 Logon       SQL Server is not ready to accept new client connections. Wait a few minutes before trying again. If you have access to the error log, look for the informational message that indicates that SQL Server is ready before trying to connect again.  [CLIENT: 10.0.0.21]
2010-10-15 11:33:47.85 Server      Server is listening on [ ::1 <ipv6> 51746].
2010-10-15 11:33:47.86 Server      Server is listening on [ 127.0.0.1 <ipv4> 51746].
2010-10-15 11:33:47.86 Server      Dedicated admin connection support was established for listening locally on port 51746.
2010-10-15 11:33:47.94 Server      The SQL Server Network Interface library could not register the Service Principal Name (SPN) for the SQL Server service. Error: 0x5, state: 4. Failure to register an SPN may cause integrated authentication to fall back to NTLM instead of Kerberos. This is an informational message. Further action is only required if Kerberos authentication is required by authentication policies.
2010-10-15 11:33:47.94 Server      SQL Server is now ready for client connections. This is an informational message; no user action is required.
2010-10-15 11:33:48.26 spid10s     Clearing tempdb database.
2010-10-15 11:33:50.58 spid13s     Error: 9003, Severity: 20, State: 1.
2010-10-15 11:33:50.58 spid13s     The log scan number (3967:240:16) passed to log scan in database 'msdb' is not valid. This error may indicate data corruption or that the log file (.ldf) does not match the data file (.mdf). If this error occurred during replication, re-create the publication. Otherwise, restore from backup if the problem results in a failure during startup.
2010-10-15 11:33:50.92 spid10s     Starting up database 'tempdb'.
2010-10-15 11:33:51.84 spid13s     Error: 3414, Severity: 21, State: 1.
2010-10-15 11:33:51.84 spid13s     An error occurred during recovery, preventing the database 'msdb' (database ID 4) from restarting. Diagnose the recovery errors and fix them, or restore from a known good backup. If errors are not corrected or expected, contact Technical Support.
2010-10-15 11:33:52.50 spid6s      Error: 8355, Severity: 16, State: 1.
2010-10-15 11:33:52.50 spid6s      Server-level event notifications can not be delivered. Either Service Broker is disabled in msdb, or msdsb failed to start. Event notifications in other databases could be affected as well. Bring msdb online, or enable Service Broker.
2010-10-15 11:33:52.87 spid13s     The Service Broker protocol transport is disabled or not configured.
2010-10-15 11:33:52.90 spid13s     The Database Mirroring protocol transport is disabled or not configured.
2010-10-15 11:33:53.49 spid13s     Service Broker manager has started.
2010-10-15 11:33:53.49 spid6s      Recovery is complete. This is an informational message only. No user action is required.
2010-10-15 11:33:55.91 Logon       Error: 18456, Severity: 14, State: 38.
2010-10-15 11:33:55.91 Logon       Login failed for user 'BACKUP\MICROSOFT$DPM$Acct'. Reason: Failed to open the explicitly specified database. [CLIENT: <local machine>]
2010-10-15 11:33:56.72 Logon       Error: 18456, Severity: 14, State: 38.
2010-10-15 11:33:56.72 Logon       Login failed for user 'BACKUP\MICROSOFT$DPM$Acct'. Reason: Failed to open the explicitly specified database. [CLIENT: <local machine>]
2010-10-15 11:33:59.82 spid51      Setting database option COMPATIBILITY_LEVEL to 100 for database ReportServer$MSDPM2010.
2010-10-15 11:34:00.22 spid51      Setting database option COMPATIBILITY_LEVEL to 100 for database ReportServer$MSDPM2010TempDB.
2010-10-15 11:34:02.34 Logon       Error: 18456, Severity: 14, State: 38.
2010-10-15 11:34:02.34 Logon       Login failed for user 'BACKUP\MICROSOFT$DPM$Acct'. Reason: Failed to open the explicitly specified database. [CLIENT: <local machine>]
2010-10-15 11:34:02.35 Logon       Error: 18456, Severity: 14, State: 38.
2010-10-15 11:34:02.35 Logon       Login failed for user 'BACKUP\MICROSOFT$DPM$Acct'. Reason: Failed to open the explicitly specified database. [CLIENT: <local
-----------------------------------------------------------------
Event Viewer in Windows is showing quite a few related errors on each reboot:
-Application Errors
- Event 999 MSDPM - The description for Event ID 999 from source MSDPM cannot be found.  Either the component that raises this event is not installed on your local computer or the installation is corrupt...
-SQLServerAgent could not be started (reason: Unable to connect to server...

System Errors:
- The DPM AccessManager Service service terminated unexpectedly.  It has done this 6 times.
- The DPM service depends on the SQL Server Agent (MSDPM2010) service which failed to start because of the following error:  The operation completed succesfully.
Marten RuneSQL Expert/Infrastructure Architect

Commented:
Well it complains about msdb and tempdb database.

Do your SQL server start? Can you use SSMS to connect? What does DBCC Checkdb give for msdb database?

//Marten

Should you be charging more for IT Services?

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

SQL Expert/Infrastructure Architect
Commented:
If it won't start, can you start it with minimal configuration?
http://msdn.microsoft.com/en-us/library/ms186400(SQL.90).aspx

//Marten

Author

Commented:
SQL Server does not start.  If you try to connect with SSMS, it mentions that the database is marked as SUSPECT! by recovery.
I tried running a query of:
dbcc checkdb (dpmdb) with no_infomsgs, all_errormsgs and didn't see any data on the bottom under messages and I am unsure if that creates an output log but I don't see a current one after I run it in the logs folder.
Marten RuneSQL Expert/Infrastructure Architect

Commented:
Quote: "SQL Server does not start", but it says something about recovery on a user database. Then it has to be started.

Lets look at it, SQL Server can host many databases. What I'm asking is if the SQL Server starts. Since it complained about msdb (wich is a system database).

Does the service 'SQL Server (MSSQLSERVER)' run or not. Type in a command window:
services.msc
and hit ENTER.
Scroll dotn to 'SQL Server (MSSQLSERVER)' and look at the status column. Does it say started? or is it blank
Next Id like to know if you have backups of the system databases, i e master, msdb and perhaps model. Do you and are they fairly recent?

Lets take it from there.

//Marten

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial