Two identical physical servers running W2012 but one has 2 errors I cannot figure out
Hi Everyone, Hoping someone can shed some light on this odd situation. I have two 100% identical physical servers (not in production) I am planning to deploy as file servers and they are both setup the same way.
DL380p G8
8 GB of Memory
Same firmware level, drivers and patches (Windows)
5 disks (300 GB 2-Port SAS Drives) in RAID 5 and 1 c: (312GB) and 1 d: (805GB) formatted disks with Shadow Copies turned on for the d: (the Shadow Copies are disabled for the c:)
Windows 2012 R2
Neither of them are doing much at the moment as stated so I struggle to understand why one of them is giving these errors.
The naughty one shows the following errors.
After I setup the Shadow Copies I started seeing this error every day (note that Shadow Copies were setup the same way as the other server that does not complain and only d: has it turned on)
- > Event ID 7001 - Source VSS
VssAdmin: Unable to create a shadow copy: Either the specified volume was not found or it is not a local volume.
Command-line: 'C:\Windows\system32\vssad
AND
- > Event ID 12290 - Source VSS
Volume Shadow Copy Service warning: GetDiskFreeSpaceEx() for \\?\Volume{401dea09-396d-1
Operation:
Check If Volume Is Supported by Provider
Context:
Execution Context: System Provider
Volume Name: \\?\Volume{401dea09-396d-1
I also noticed this error periodically (and that is the only other error on that server)
- > Event ID 906 - Source ESENT
svchost (500) A significant portion of the database buffer cache has been written out to the system paging file. This may result in severe performance degradation.
See help link for complete details of possible causes.
Resident cache has fallen by 4832 buffers (or 100) in the last 940759 seconds.
Current Total Percent Resident: 0 (0 of 4832 buffers)
I note that the server is not doing any backups and has no backup software installed yet.
Also when doing a vssadmin list writers no errors show up (see end of this page)
I have reformatted (fast and regular) the d: drive twice now to see if resetting it helps, but the errors come back.
The array health and disk health (all 5) is good and I don't see any hardware related failures.
Controller on System Board
Controller Status OK
Model HP Smart Array P420i Controller
Cache Module Status OK
Cache Module Memory 524288 KB
-Drive Enclosure Port 1I Box 2
Status OK
Drive Bays 4
-Drive Enclosure Port 2I Box 2
Status OK
Drive Bays 4
-Physical Drive in Port 1I Box 2 Bay 1
Status OK
Model EH0300FCBVC
Capacity 279 GB
Location Port 1I Box 2 Bay 1
Drive Configuration Configured
-Physical Drive in Port 1I Box 2 Bay 2
Status OK
Model EH0300FCBVC
Capacity 279 GB
Location Port 1I Box 2 Bay 2
Drive Configuration Configured
-Physical Drive in Port 1I Box 2 Bay 3
Status OK
Model EH0300FCBVC
Capacity 279 GB
Location Port 1I Box 2 Bay 3
Drive Configuration Configured
-Physical Drive in Port 1I Box 2 Bay 4
Status OK
Model EH0300FBQDD
Capacity 279 GB
Location Port 1I Box 2 Bay 4
Drive Configuration Configured
-Physical Drive in Port 2I Box 2 Bay 5
Status OK
Model EH0300FBQDD
Capacity 279 GB
Location Port 2I Box 2 Bay 5
Drive Configuration Configured
I also see no errors or other issues with the memory
Memory Summary
Processor 1 12 4 GB 1333 MHz 1.35 V
Processor 2 12 4 GB 1333 MHz 1.35 V
Processor 1 12 Good, In Use Yes N/A DIMM DDR3 4096 MB 1333 MHz 1.35 V 1 RDIMM
Processor 2 12 Good, In Use Yes N/A DIMM DDR3 4096 MB 1333 MHz 1.35 V 1 RDIMM
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2012 Microsoft Corp.
Writer name: 'Task Scheduler Writer'
Writer Id: {d61d61c8-d73a-4eee-8cdd-f
Writer Instance Id: {1bddd48e-5052-49db-9b07-b
State: [1] Stable
Last error: No error
Writer name: 'VSS Metadata Store Writer'
Writer Id: {75dfb225-e2e4-4d39-9ac9-f
Writer Instance Id: {088e7a7d-09a8-4cc6-a609-a
State: [1] Stable
Last error: No error
Writer name: 'Performance Counters Writer'
Writer Id: {0bada1de-01a9-4625-8278-6
Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a
State: [1] Stable
Last error: No error
Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1
Writer Instance Id: {089ffca5-20c4-4a5b-9430-8
State: [1] Stable
Last error: No error
Writer name: 'ASR Writer'
Writer Id: {be000cbe-11fe-4426-9c58-5
Writer Instance Id: {0e6b7750-c72b-4266-9801-3
State: [1] Stable
Last error: No error
Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-7
Writer Instance Id: {099789b4-54d2-4ff6-a9ba-5
State: [1] Stable
Last error: No error
Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7
Writer Instance Id: {966f64a8-a29d-40c6-931b-8
State: [1] Stable
Last error: No error
Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-4
Writer Instance Id: {a444e8ea-f1fd-4bdb-b1b5-6
State: [1] Stable
Last error: No error
Writer name: 'Shadow Copy Optimization Writer'
Writer Id: {4dc3bdd4-ab48-4d07-adb0-3
Writer Instance Id: {f8bf4e0e-061c-4b83-a211-d
State: [1] Stable
Last error: No error
Any suggestions? Much appreciated :-)
DL380p G8
8 GB of Memory
Same firmware level, drivers and patches (Windows)
5 disks (300 GB 2-Port SAS Drives) in RAID 5 and 1 c: (312GB) and 1 d: (805GB) formatted disks with Shadow Copies turned on for the d: (the Shadow Copies are disabled for the c:)
Windows 2012 R2
Neither of them are doing much at the moment as stated so I struggle to understand why one of them is giving these errors.
The naughty one shows the following errors.
After I setup the Shadow Copies I started seeing this error every day (note that Shadow Copies were setup the same way as the other server that does not complain and only d: has it turned on)
- > Event ID 7001 - Source VSS
VssAdmin: Unable to create a shadow copy: Either the specified volume was not found or it is not a local volume.
Command-line: 'C:\Windows\system32\vssad
AND
- > Event ID 12290 - Source VSS
Volume Shadow Copy Service warning: GetDiskFreeSpaceEx() for \\?\Volume{401dea09-396d-1
Operation:
Check If Volume Is Supported by Provider
Context:
Execution Context: System Provider
Volume Name: \\?\Volume{401dea09-396d-1
I also noticed this error periodically (and that is the only other error on that server)
- > Event ID 906 - Source ESENT
svchost (500) A significant portion of the database buffer cache has been written out to the system paging file. This may result in severe performance degradation.
See help link for complete details of possible causes.
Resident cache has fallen by 4832 buffers (or 100) in the last 940759 seconds.
Current Total Percent Resident: 0 (0 of 4832 buffers)
I note that the server is not doing any backups and has no backup software installed yet.
Also when doing a vssadmin list writers no errors show up (see end of this page)
I have reformatted (fast and regular) the d: drive twice now to see if resetting it helps, but the errors come back.
The array health and disk health (all 5) is good and I don't see any hardware related failures.
Controller on System Board
Controller Status OK
Model HP Smart Array P420i Controller
Cache Module Status OK
Cache Module Memory 524288 KB
-Drive Enclosure Port 1I Box 2
Status OK
Drive Bays 4
-Drive Enclosure Port 2I Box 2
Status OK
Drive Bays 4
-Physical Drive in Port 1I Box 2 Bay 1
Status OK
Model EH0300FCBVC
Capacity 279 GB
Location Port 1I Box 2 Bay 1
Drive Configuration Configured
-Physical Drive in Port 1I Box 2 Bay 2
Status OK
Model EH0300FCBVC
Capacity 279 GB
Location Port 1I Box 2 Bay 2
Drive Configuration Configured
-Physical Drive in Port 1I Box 2 Bay 3
Status OK
Model EH0300FCBVC
Capacity 279 GB
Location Port 1I Box 2 Bay 3
Drive Configuration Configured
-Physical Drive in Port 1I Box 2 Bay 4
Status OK
Model EH0300FBQDD
Capacity 279 GB
Location Port 1I Box 2 Bay 4
Drive Configuration Configured
-Physical Drive in Port 2I Box 2 Bay 5
Status OK
Model EH0300FBQDD
Capacity 279 GB
Location Port 2I Box 2 Bay 5
Drive Configuration Configured
I also see no errors or other issues with the memory
Memory Summary
Processor 1 12 4 GB 1333 MHz 1.35 V
Processor 2 12 4 GB 1333 MHz 1.35 V
Processor 1 12 Good, In Use Yes N/A DIMM DDR3 4096 MB 1333 MHz 1.35 V 1 RDIMM
Processor 2 12 Good, In Use Yes N/A DIMM DDR3 4096 MB 1333 MHz 1.35 V 1 RDIMM
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2012 Microsoft Corp.
Writer name: 'Task Scheduler Writer'
Writer Id: {d61d61c8-d73a-4eee-8cdd-f
Writer Instance Id: {1bddd48e-5052-49db-9b07-b
State: [1] Stable
Last error: No error
Writer name: 'VSS Metadata Store Writer'
Writer Id: {75dfb225-e2e4-4d39-9ac9-f
Writer Instance Id: {088e7a7d-09a8-4cc6-a609-a
State: [1] Stable
Last error: No error
Writer name: 'Performance Counters Writer'
Writer Id: {0bada1de-01a9-4625-8278-6
Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a
State: [1] Stable
Last error: No error
Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1
Writer Instance Id: {089ffca5-20c4-4a5b-9430-8
State: [1] Stable
Last error: No error
Writer name: 'ASR Writer'
Writer Id: {be000cbe-11fe-4426-9c58-5
Writer Instance Id: {0e6b7750-c72b-4266-9801-3
State: [1] Stable
Last error: No error
Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-7
Writer Instance Id: {099789b4-54d2-4ff6-a9ba-5
State: [1] Stable
Last error: No error
Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7
Writer Instance Id: {966f64a8-a29d-40c6-931b-8
State: [1] Stable
Last error: No error
Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-4
Writer Instance Id: {a444e8ea-f1fd-4bdb-b1b5-6
State: [1] Stable
Last error: No error
Writer name: 'Shadow Copy Optimization Writer'
Writer Id: {4dc3bdd4-ab48-4d07-adb0-3
Writer Instance Id: {f8bf4e0e-061c-4b83-a211-d
State: [1] Stable
Last error: No error
Any suggestions? Much appreciated :-)
4GB Memory is nowhere near enough memory. Oops, I see you have 8GB I'd still double up the memory.
Command-line: 'C:\Windows\system32\vssad
you are using the GUID (globally unique Identifier) which is incorrect use the drive letter instead.
Command-line: 'C:\Windows\system32\vssad
you are using the GUID (globally unique Identifier) which is incorrect use the drive letter instead.
David, I just checked the created scheduled task for my own server - it uses GUIDs, not drive letters. I think the first step here is to figure out what drive letter is being processed. If you can narrow down my powershell command (which should list a ton of drive info) to just the two fields necessary, otherwise Laszlo Denes, you can review all the output information and determine which drive it's having an issue with.
Also, David, I disagree. A file server doesn't need more than 2 GB of RAM in MOST cases. I use dynamic RAM with my small business DC/File Server combos and while I allow up to 4, I rarely go past 2 GB. And that's with DC functionality.
Laszlo Denes, these should be VMs. I hope they are. Though the RAM is low on the host, it would be sufficient for a single VM file server. Everything should be virtual these days unless you have a good reason for them not to be.
Also, David, I disagree. A file server doesn't need more than 2 GB of RAM in MOST cases. I use dynamic RAM with my small business DC/File Server combos and while I allow up to 4, I rarely go past 2 GB. And that's with DC functionality.
Laszlo Denes, these should be VMs. I hope they are. Though the RAM is low on the host, it would be sufficient for a single VM file server. Everything should be virtual these days unless you have a good reason for them not to be.
ASKER
Lee/David - They are not VM but two physical servers and the 8GB ram ought to be enough as we run another file server with only 4 GB (2008 r2) on a lower end system and it works fine. However, I was also poking around and noted that perhaps this occurred on the one that is misbehaving during testing.
Before deleting a volume that is being shadow copied, delete the scheduled task for creating shadow copies. If the volume is deleted without deleting the shadow copy task, the scheduled task will fail and an Event ID: 7001 error will be written to the event log. Delete the task before deleting the volume to avoid filling the event log with these errors. To manually delete the scheduled task, click Start , point to Administrative Tools , and then click Task Scheduler . In Task Scheduler, click Task Scheduler Library , right-click the task to create shadow copies, and then click Delete .
If that is the case then removing the offending tasks should fix it and then redoing the shadow copies.... right now the drive has been cleanly formatted and there are no more error messages since then, but I have also not turned the shadow copy back on yet...
However, I cannot for the life of me figure what the task is called in task scheduler so I can delete it and I don't see anything similar, even on the 2012 server that does have working shadow copies (I have verified they work)... I can see the task under the summary, but cannot delete it from there... and in the case of the naughty server (that does not have shadow copies set right now so there should be no tasks) I can see two such tasks... the other one (the server that works) only has one and that makes sense --- see screenshot BUT WHERE DO I DELETE THEM... LOL
Before deleting a volume that is being shadow copied, delete the scheduled task for creating shadow copies. If the volume is deleted without deleting the shadow copy task, the scheduled task will fail and an Event ID: 7001 error will be written to the event log. Delete the task before deleting the volume to avoid filling the event log with these errors. To manually delete the scheduled task, click Start , point to Administrative Tools , and then click Task Scheduler . In Task Scheduler, click Task Scheduler Library , right-click the task to create shadow copies, and then click Delete .
If that is the case then removing the offending tasks should fix it and then redoing the shadow copies.... right now the drive has been cleanly formatted and there are no more error messages since then, but I have also not turned the shadow copy back on yet...
However, I cannot for the life of me figure what the task is called in task scheduler so I can delete it and I don't see anything similar, even on the 2012 server that does have working shadow copies (I have verified they work)... I can see the task under the summary, but cannot delete it from there... and in the case of the naughty server (that does not have shadow copies set right now so there should be no tasks) I can see two such tasks... the other one (the server that works) only has one and that makes sense --- see screenshot BUT WHERE DO I DELETE THEM... LOL
ASKER
Got the error again and there are NO Shadow Copies setup as I reformatted that drive.
So it must be trying one that does not exist. How do I delete the task from the task scheduler since I can see that it is doing it, but cannot find the actual task itself (see screenshot) so I can delete it.
VssAdmin: Unable to create a shadow copy: Either the specified volume was not found or it is not a local volume.
Command-line: 'C:\Windows\system32\vssad
shadowcopytask.jpg
So it must be trying one that does not exist. How do I delete the task from the task scheduler since I can see that it is doing it, but cannot find the actual task itself (see screenshot) so I can delete it.
VssAdmin: Unable to create a shadow copy: Either the specified volume was not found or it is not a local volume.
Command-line: 'C:\Windows\system32\vssad
shadowcopytask.jpg
Have you looked at the tasks within Disk Management?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
That did the trick perfectly. Once removed the old instance the error did not return!
Powershell:
GWMI -namespace root\cimv2 -class win32_volume