We have an SBS11 STD HP box at a site and it has started for no obvious reason to increase the Virtual Memory size rapidly over a period of about 24 hours. This goes up and up until the Exchange Transport service stops and things start to fall over.
I'm trying to traced what is causing this and am using VMMap which shows svchost.exe taking up way more than it should. This of course contains a lot of running processes, but I'm finding hard to decipher if these processes inside SVC host are causing this or if it's just ballooning itself. It could be something else as well, however the other server's I've checked have no where near the large svchost usage in the Virtual Memory.
The standard memory or CPU doesn't have a large svchost process, just the virtual memory. I've confirmed this increasing virtual memory use with perfmon and the graph spike up at a dramatic rate over 24 hours. The server has 12GB Ram and the Page file is 22gb. There was a program added very recently that downloads pathology results for this DR surgery and this requires the Server to be logged on all the time (until we can get it to work as a service.!) Also, recently the 2nd NIC was enabled and then disabled for a few minutes which isn't great to do in SBS I've heard. Finally we have had issues with a third party monitoring software agent causing WSUS problems and I've stopped the WSUS IIS application pool at the moment to also free some RAM while we get this sorted.
I've attached a screencap of the current VMMap screen inside the svchost.exe process which shows how much it's using at the moment.