stevenmholder
asked on
wbadmin.msc fails to start
Log shows "The Block Level Backup Engine Service service terminated with the following error:
%%-2147024713"
Tried running wbadmin get status under cmd but get "Server execution failed" no matter the syntax.
Which is the same error under the GUI. Restarted the server no difference.
%%-2147024713"
Tried running wbadmin get status under cmd but get "Server execution failed" no matter the syntax.
Which is the same error under the GUI. Restarted the server no difference.
The service is crashing; do you have any other software for backup purposes installed? What antivirus are you running, and does what errors does the Event Viewer contain regarding Windows Backup, or that service?
ASKER
We are running MozyPro on the server also for off-site backup. It starts at 1 am. WS backup starts at 11:30 pm. Shadow copy service is running.
Prior to this failure we were getting warnings.. "The backup operation that started at '¿2010¿-¿10¿-¿07T03:30:20.
Now the Application log showsInformation.."The Block Level Backup Engine service has stopped." at the time of scheduled backup followed by an error in the log that shows.."The backup operation that started at '¿2010¿-¿10¿-¿09T03:30:30.
We are also running a program called WatchDirectory but none of these programs were changed when the WS Backup started to fail on 10/09/2010.
Prior to this failure we were getting warnings.. "The backup operation that started at '¿2010¿-¿10¿-¿07T03:30:20.
Now the Application log showsInformation.."The Block Level Backup Engine service has stopped." at the time of scheduled backup followed by an error in the log that shows.."The backup operation that started at '¿2010¿-¿10¿-¿09T03:30:30.
We are also running a program called WatchDirectory but none of these programs were changed when the WS Backup started to fail on 10/09/2010.
Nothing is readily apparent from those errors unfortunately.
Have any updates been installed since then? I would try rebooting, if that doesn't work, I would try reinstalling the Backup feature.
Does the System Log show anything related?
Have any updates been installed since then? I would try rebooting, if that doesn't work, I would try reinstalling the Backup feature.
Does the System Log show anything related?
ASKER
System log just shows "The Block Level Backup Engine Service service entered the stopped state." at the time of scheduled backup.
How do I reinstall backup? I cannot find the way to uninstall.
How do I reinstall backup? I cannot find the way to uninstall.
Go into Server Manager > Features > Remove Features (Right Hand side) > uncheck 'Windows Server Backup'
Were any Windows updates done since it started crashing?
Were any Windows updates done since it started crashing?
ASKER
Thanks for staying with me on this one.. I tried removing and adding but it still failed.
The one update that installed on October 8 was "Security Update for Microsoft .NET Framework 3.5.1, Windows 7, and Windows Server 2008 R2 for x64-based Systems (KB2416471)"
Installation date: ¿10/¿8/¿2010 3:00 AM
Installation status: Successful
Update type: Important
A security issue has been identified that could allow an attacker to compromise your Windows-based system that is running the Microsoft .NET Framework and gain access to information. You can help protect your computer by installing this update from Microsoft. After you install this item, you may have to restart your computer.
More information:
http://go.microsoft.com/fwlink/?LinkId=202373
Help and Support:
http://support.microsoft.com
"
The one update that installed on October 8 was "Security Update for Microsoft .NET Framework 3.5.1, Windows 7, and Windows Server 2008 R2 for x64-based Systems (KB2416471)"
Installation date: ¿10/¿8/¿2010 3:00 AM
Installation status: Successful
Update type: Important
A security issue has been identified that could allow an attacker to compromise your Windows-based system that is running the Microsoft .NET Framework and gain access to information. You can help protect your computer by installing this update from Microsoft. After you install this item, you may have to restart your computer.
More information:
http://go.microsoft.com/fwlink/?LinkId=202373
Help and Support:
http://support.microsoft.com
"
My pleasure.
That update shouldn't affect the backup services at all. Are you running any antivirus on the server, and are there any power outages, or other spurious errors since it started failing?
Any new hardware installed on the server (drivers loaded)?
That update shouldn't affect the backup services at all. Are you running any antivirus on the server, and are there any power outages, or other spurious errors since it started failing?
Any new hardware installed on the server (drivers loaded)?
ASKER
I know, it's very strange. The server is running on an APC battery backup but there were no outages. No new hardware, nothing. The only change was the .NET update I showed before.I really want a cold metal type backup running on this server. Any suggestions?
ASKER
No antivirus.
Please post the logs from:
C:\Windows\Logs\WindowsSer
C:\Windows\Logs\WindowsSer
ASKER
I have posted a screen shot of the log folder, logwindow.jpg and the 2 logs from 10/8/2010 and the 2 ..etl files that follows on 10/11/2010. Do not know where the logs from 10/9 and 10/10 went..??? Also added another type log file..UI?
They are all in the zipped folder attached. I had to rename the .etl files with ..doc as .etl was not allowed in the upload to EE. Note these are .etl files NOT Word documnets, please change extension after you download.
WB-Issue.zip
They are all in the zipped folder attached. I had to rename the .etl files with ..doc as .etl was not allowed in the upload to EE. Note these are .etl files NOT Word documnets, please change extension after you download.
WB-Issue.zip
Is MozyPro the most up to date version? I am suspecting an error in their software. I would first try uninstalling it, rebooting, and installing the most up to date version.
The errors are so far that I see, are only on the D: drive, and it's shadow copies.
How much data is on the D: drive, how much free space is there, and how much space are the shadow copies configured to use?
Run checkdisk 'chkdsk D: /R' overnight (reboot, and let it run if there are any in use files) See if it repairs any errors.
Try to backup the C: drive instead of D: and see if you get the same errors.
I would also try an external usb / drive, and run wbadmin against it. This will separate a disk problem, or a windows backup problem.
If none of those work, then I would do the following, one by one trying to backup after each:
Uninstall Mozy Pro completely, and reboot.
Disable Shadow copies, and System Restore on D: (assuming C: backs up okay.)
Please also post the exact command / script / configuration (if using the gui) that calls wbadmin.
The errors are so far that I see, are only on the D: drive, and it's shadow copies.
How much data is on the D: drive, how much free space is there, and how much space are the shadow copies configured to use?
Run checkdisk 'chkdsk D: /R' overnight (reboot, and let it run if there are any in use files) See if it repairs any errors.
Try to backup the C: drive instead of D: and see if you get the same errors.
I would also try an external usb / drive, and run wbadmin against it. This will separate a disk problem, or a windows backup problem.
If none of those work, then I would do the following, one by one trying to backup after each:
Uninstall Mozy Pro completely, and reboot.
Disable Shadow copies, and System Restore on D: (assuming C: backs up okay.)
Please also post the exact command / script / configuration (if using the gui) that calls wbadmin.
ASKER
This is a remote server for me and I need to be there to reboot (halts at F2) so I will have to update next week. I was able to capture the attached screen shot.. Files services are not running?
file-services.jpg
file-services.jpg
Can you export, and upload your event logs? Specifically Application, and System. It looks like there are some more errors there that will help diagnose this.
ASKER
Here they are.. BTW installed Second copy as an interim backup solution until I resolve this. Log extensions chaged from .evtx to txt because EE does not allow evtx extension.
sytemlog.zip
sytemlog.zip
The wdpostman Service seems to crash with Event ID 7023 after the system reboots.
On 9/29 KB2158563 was installed at 12:45 PM. The server rebooted, it came back up at 12:59. The clock here jumps from 12:45 to 12:59, then back to 12:45 again. It comes back up at 1:02:39. It shuts down again at 1:07:58, and comes back up by 1:14:05. It doesn't show a user doing this, but this is the day that your issues start.
Between 11:40 PM on 9/29, and 2:30 AM on 9/30 there are a slew of 'controller errors':
The driver detected a controller error on \Device\Ide\IdePort2. (The 2nd IDE port)
This correlates to the first backup operation that actually failed (albeit 10 minutes later) with error code '2155347997'.
Also at 11:31 PM on 9/30 through 10/5 the following is shown:
Event ID 55: Source: Ntfs:
The file system structure on the disk is corrupt and unusable. Please run the chkdsk utility on the volume \\?\Volume{d4177daf-c822-1
To find out which Guid that disk is referencing run the command 'mountvol' without any arguments.
Ultimately it looks to me like Your hard drive is corrupt. If that is the C: drive, and checkdisk doesn't fix it, you will need to reformat / restore from backup / replace the server. If it is D:, then you may be okay, but you will need to do more troubleshooting to see if you can run the backup on another Volume.
Also, there are a slew of these events:
Event ID 5774:
The dynamic registration of the DNS record 'your_domain.com. 600 IN A x.x.x.x' failed on the following DNS server:
DNS server IP address: x.x.x.x
Returned Response Code (RCODE): 5
Returned Status Code: 9017
For computers and users to locate this domain controller, this record must be registered in DNS.
USER ACTION
Determine what might have caused this failure, resolve the problem, and initiate registration of the DNS records by the domain controller. To determine what might have caused this failure, run DCDiag.exe. To learn more about DCDiag.exe, see Help and Support Center. To initiate registration of the DNS records by this domain controller, run 'nltest.exe /dsregdns' from the command prompt on the domain controller or restart Net Logon service.
Or, you can manually add this record to DNS, but it is not recommended.
This is your Domain Controller trying to register Service records on a dns server, the dns server seems to be external to your network. I don't know how it is setup, but unless you have a partner organization, or other domain located here, that attempted propagation needs to either be fixed, or blown away.
Other pertinent events:
9-25
9-30
10-7
10-8
The backup operation that started at '¿2010¿-¿09¿-¿29T03:30:22.
9-30
10-5
10-8
10-14
The backup operation that started at '¿2010¿-¿10¿-¿01T03:30:13.
These two errors occur intermittently between 10:35, and 11:47 PM, only on 10/15/2010:
Event ID 8193:
Volume Shadow Copy Service error: Unexpected error calling routine CoCreateInstance. hr = 0x80040154, Class not registered
.
Operation:
Instantiating VSS server
Event ID 22:
Volume Shadow Copy Service error: A critical component required by the Volume Shadow Copy service is not registered. This might happened if an error occurred during Windows setup or during installation of a Shadow Copy provider. The error returned from CoCreateInstance on class with CLSID {e579ab5f-1cc4-44b4-bed9-d
].
Operation:
Instantiating VSS server
On 9/29 KB2158563 was installed at 12:45 PM. The server rebooted, it came back up at 12:59. The clock here jumps from 12:45 to 12:59, then back to 12:45 again. It comes back up at 1:02:39. It shuts down again at 1:07:58, and comes back up by 1:14:05. It doesn't show a user doing this, but this is the day that your issues start.
Between 11:40 PM on 9/29, and 2:30 AM on 9/30 there are a slew of 'controller errors':
The driver detected a controller error on \Device\Ide\IdePort2. (The 2nd IDE port)
This correlates to the first backup operation that actually failed (albeit 10 minutes later) with error code '2155347997'.
Also at 11:31 PM on 9/30 through 10/5 the following is shown:
Event ID 55: Source: Ntfs:
The file system structure on the disk is corrupt and unusable. Please run the chkdsk utility on the volume \\?\Volume{d4177daf-c822-1
To find out which Guid that disk is referencing run the command 'mountvol' without any arguments.
Ultimately it looks to me like Your hard drive is corrupt. If that is the C: drive, and checkdisk doesn't fix it, you will need to reformat / restore from backup / replace the server. If it is D:, then you may be okay, but you will need to do more troubleshooting to see if you can run the backup on another Volume.
Also, there are a slew of these events:
Event ID 5774:
The dynamic registration of the DNS record 'your_domain.com. 600 IN A x.x.x.x' failed on the following DNS server:
DNS server IP address: x.x.x.x
Returned Response Code (RCODE): 5
Returned Status Code: 9017
For computers and users to locate this domain controller, this record must be registered in DNS.
USER ACTION
Determine what might have caused this failure, resolve the problem, and initiate registration of the DNS records by the domain controller. To determine what might have caused this failure, run DCDiag.exe. To learn more about DCDiag.exe, see Help and Support Center. To initiate registration of the DNS records by this domain controller, run 'nltest.exe /dsregdns' from the command prompt on the domain controller or restart Net Logon service.
Or, you can manually add this record to DNS, but it is not recommended.
This is your Domain Controller trying to register Service records on a dns server, the dns server seems to be external to your network. I don't know how it is setup, but unless you have a partner organization, or other domain located here, that attempted propagation needs to either be fixed, or blown away.
Other pertinent events:
9-25
9-30
10-7
10-8
The backup operation that started at '¿2010¿-¿09¿-¿29T03:30:22.
9-30
10-5
10-8
10-14
The backup operation that started at '¿2010¿-¿10¿-¿01T03:30:13.
These two errors occur intermittently between 10:35, and 11:47 PM, only on 10/15/2010:
Event ID 8193:
Volume Shadow Copy Service error: Unexpected error calling routine CoCreateInstance. hr = 0x80040154, Class not registered
.
Operation:
Instantiating VSS server
Event ID 22:
Volume Shadow Copy Service error: A critical component required by the Volume Shadow Copy service is not registered. This might happened if an error occurred during Windows setup or during installation of a Shadow Copy provider. The error returned from CoCreateInstance on class with CLSID {e579ab5f-1cc4-44b4-bed9-d
].
Operation:
Instantiating VSS server
ASKER
Thanks RedWraith94 I am going to the account tomorrow and will try your suggestions.
ASKER
Run chkdsk on C and D but Windows backup will simply not start. This is a production environment and we just rebuilt the server a couple months ago so restoring again is not a great option but looks like it might have to be. Unless anyone can give any more suggestions in the next week I will close the issue. See attached screen shot to see exact issue.
application-error.jpg
application-error.jpg
Have the controller errors come back up in the eventlog? If it is a bad hard drive array controller (ide in this case), or a bad disk rebuilding on the same hardware will only delay the onset of this, or similar errors recurring.
Does the server reboot by itself ever, and are there any files in the C:\Windows\Minidump directory? (BSOD crash dumps).
Does the server reboot by itself ever, and are there any files in the C:\Windows\Minidump directory? (BSOD crash dumps).
ASKER
No more controller errors. No unscheduled server reboots. We are considering Acronis.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Windows can't always be fixed, forum environments add their own level of difficulty. I would still recommend xcopy / robocopy for cost. Unless you need the Bare Metal Recovery options that Acronis offers.
I am still suspect of those disks, or that controller card, motherboard, power supply etc. Good luck with it.
I am still suspect of those disks, or that controller card, motherboard, power supply etc. Good luck with it.
ASKER
We purchased Acronis for $853 Expensive fix.
ASKER
Don't understand why I cannot close this. ????
ASKER
We purchased Acronis for $853 Expensive fix.