ZenQuest
asked on
Virtual machine file server SLLLLOW after trying to back it up with Symantec Backup Exec
I'm not sure where to begin with this problem but I'm losing my mind!!!!
So we have vSphere 4. I have a Windows 2003 Enterprise VM that serves as a file server for My Documents / Application Data redirection through active directory. That partition is around 1TB and has nearly 1 million files in it. Up until the other day it worked great...
All started like this... I installed the Symantec Backup Exec Remote Windows Agent on this machine. We have Symantec Vmware Agent on our Backup Exec Server. For GRT to work, the agent must be installed on the client. This is all Symantec Backup Exec 2010 R3 stuff.
So... I set up a job in backup exec to backup this server. It ran for 48 hours before failing. I noticed in vcenter that it had created a snapshot / removed a snapshot.. etc. etc. The error code in BUE was "Unable to copy the virtual machine disk using the VMware VixDiskLib".
So now... this server is HORRIBLY SLOW. The CPU stays pegged out all the time inside vcenter and inside the VM. Rebooting does nothing to help. I've uninstalled the remote backup agent. I've uninstalled endpoint protection. Inside vcenter if I right click the VM, migrate is grayed out. I can't drag and drop to another server. All other servers are running fine alongside this one.
After hours when this server is idle.. CPU is still high. Ping responses are slow.
I'm convinced Symantec Backup Exec has done somethign to the VM inside vcenter. But I just don't know where to look.
Does anyone have ANY ideas?
Thanks!
So we have vSphere 4. I have a Windows 2003 Enterprise VM that serves as a file server for My Documents / Application Data redirection through active directory. That partition is around 1TB and has nearly 1 million files in it. Up until the other day it worked great...
All started like this... I installed the Symantec Backup Exec Remote Windows Agent on this machine. We have Symantec Vmware Agent on our Backup Exec Server. For GRT to work, the agent must be installed on the client. This is all Symantec Backup Exec 2010 R3 stuff.
So... I set up a job in backup exec to backup this server. It ran for 48 hours before failing. I noticed in vcenter that it had created a snapshot / removed a snapshot.. etc. etc. The error code in BUE was "Unable to copy the virtual machine disk using the VMware VixDiskLib".
So now... this server is HORRIBLY SLOW. The CPU stays pegged out all the time inside vcenter and inside the VM. Rebooting does nothing to help. I've uninstalled the remote backup agent. I've uninstalled endpoint protection. Inside vcenter if I right click the VM, migrate is grayed out. I can't drag and drop to another server. All other servers are running fine alongside this one.
After hours when this server is idle.. CPU is still high. Ping responses are slow.
I'm convinced Symantec Backup Exec has done somethign to the VM inside vcenter. But I just don't know where to look.
Does anyone have ANY ideas?
Thanks!
Check VM Properties Right Click the VM, Select Snapshot, Snapshot Manager, and check if there is a Snapshot Present?
ASKER
That's exactly what I thought.. in fact, this happened before and deleting the snapshot fixed the issue. Snapshot manager shows no snapshots though.
Is it possible vcenter isn't aware that its in a snapshot state?
Is it possible vcenter isn't aware that its in a snapshot state?
It is possible. You can use vCenter to browse the data stores. Select Datastores from the View _> Inventory menu and then locate the data tore that the server is using and browse it and see if there are snapshot files in the folder holding the VMDK and other files associated with that VM. If there are, sometimes you can create another snapshot in the snapshot manager and then choose to delete all snaps and it will roll back through them all. You will have to have enough disk space on that LUN to allow the snapshot process to roll back and if there are a lot of them or a really large one, it could take quite a while.If you find this is the case and have any support from vmWare or a third party you may also want to engage them.
ASKER
I've embedded a screenshot of the VM's datastore. It has a 10gig and 600gig partition... I was a little high when I said close to a TB. I've grown that 600gig partition from 250 to 350 to 600 over the last two years so Windows show alls the different partitions making up the volume. I did notice that on ANOTHER datastore I have a UserData folder with a 100Gig VMDK file in it (only file in directory). I'm thinking I created that drive a long time ago and deleted it but it still remains on the datastore. The date on the file is May, 2010. For some reason, today, the migrate option is accessible but I haven't tried doing anything as this is a production server. I'm tempted to 'clone' this server and see if the clone runs better... or maybe provision a new server and copy all the data from the old to the new... I'm running out of thigns to check.
No snapshot present in that folder. But it is possible snapshots can exist but not listed, but not this time.
What services are running in VM, and what does task manager show?
What services are running in VM, and what does task manager show?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
It sounds like you have created a snapshot, left it there for a long period of time and now your VM is deleting that snapshot still. Just a possible thought??