SBS 2008 Hangs and has to hard booted. Need help fast.

We have a SBS 2008 server that is being used as second server.  Mainly just a file share and a webserver for a LOB app we use.  The server is hanging with 99% memory usage and becoming unresponsive.  Task manager and Resource manager don't show any programs using that much memory.  The system has 32GB.   I've ran RAMMAP and the File Summary, the top file, using nearly 28GB of RAM is the Shadowprotect backup file on an external hard drive.     This problem only seems to happen when a USB drive is plugged in.   If I unplug the USB drive and run backups to a file share on another server, there's no issue.

For reference, Exchange is not running and has been removed from this server.  It's not a domain controller.  

Any ideas?
Matt W.Asked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Have you contacted Storagecraft about this?  It certainly sounds like an issue with their Shadowprotect software.

If you shut off Shadowprotect and reconnect the USB drive, is there a problem with RAM usage?
An SBS server has to have Exchange running, and it also has to be the main DC of your domain. You can't just remove the roles it needs, or use it as a "2nd" server. Otherwise you will be using it against the license agreement of m$, and it will also be unstable, and the issues you are having could well be caused by this.

If you don't use the SBS the normal way, just shut it down and replace it with a 2008 r2 standard server, or a 2012 r2 standard server.
Cris HannaSr IT Support EngineerCommented:
Rindi is right on the money here.  Your SBS Server is running in an unsupported configuration.   While you CAN uninstall Exchange from SBS its not recommended (just disable its services) but it MUST be the First Server in the AD Forest/Domain and hold all FSMO roles to be in compliance.

Sounds like potentially there is a driver issue for your USB connection.
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

In fact the only reason exchange can be uninstalled from an SBS server is so you can migrate the SBS server along with exchange to another Exchange server inside your Domain and keep your AD clean. After exchange has been removed from the SBS server you are expected to completely decommission the SBS server and not use it anymore, or set it up again as a new complete SBS server with all it's roles, in it's own, new Domain.
Cris HannaSr IT Support EngineerCommented:
Actually I would have to disagree with you there.  It can be uninstalled, but breaks some of the wizards.  But don't really think that the existence of exchange affects the is in this post
If you uninstall it and keep on using the server you are using the server against the terms of m$'s license agreement, and you basically agree to those terms when you use the server.
Matt W.Author Commented:
Guys I appreciate your concerns on the licensing.  I've informed our customer about this, and there is a roadmap to make the appropriate changes.  However, at this point we have get our actual issue resolved, so if we can focus on that, that would be great.  

To give a bit more information.   Last night I performed a test, I moved the external backup destination drive over to a different machine on the network and shared it.  I then performed a backup to that share.  When I started the backup, the machine that hosted the share RAM usage pegged at 99% and the machine came to a crawl.  This continued until I went to disk management and changed this destination disk to offline.   While it was pegged I ran RamMap on the machine and just like on the server, a shadowprotect image was taking up every bit of free RAM in the file summary tab of RamMap.   The odd part is, the shadowprotect file that was filling up RAM was NOT the volume currently being backed up.  The backup was for the C: drive and the file filling RAM was the D: drive backup file.  I'm thoroughly confused at this point.  

So I'm thinking it may not be USB related, but much more Shadowprotect related.  I've got a ticket open with them and we'll see what comes of it.    Any help in the meantime to get this resolved will be greatly appreciated.
Cris HannaSr IT Support EngineerCommented:
Have you turned off the SBS Backup jobs?
Matt W.Author Commented:
They've never been turned on.
Cris HannaSr IT Support EngineerCommented:
Are you running the SBS specific version of ShadowProtect?   What version are you running?
Matt W.Author Commented:
We're running the MSP version.  Shadow protect's agent is the same across all Windows platforms.  Only the licensing is different.  

I removed Shadow Protect and Image manager from this server, rebooted and reinstalled.  There has been an update for Image manager in the last week so I installed the newest version.  

I attached the USB directly to the server again last night and reconfigured backups and ran them.  They completed as scheduled along with the incremental backups so far this morning.   During this time, RAM usage never peaked above 8GB on this server.    At this point I have not started managing the backup folder with Image Manager yet.  My suspicion is that Image Manager might actually be the process that is loading that backup volume into RAM.   I'm going to let the backups run the rest of the day and try IM tonight.
Matt W.Author Commented:
I've confirmed Image Manager as the source, but still not sure why.   Backups complete fine and all is well in the world until you add the destination folder to be managed by Image Manager.   Once you do this, RAM usage immediately starts to climb and eventually after a few minutes pegs at 99% and hangs the server.  RamMap confirms in the file summary tab that the backup file is loaded in RAM.   You can click "stop managing folder" in Image Manager and the RAM usage immediately drops back to idle.  

Curiouser and curiouser.
Matt W.Author Commented:
I may have something....I'm testing over the weekend and will follow up.
Matt W.Author Commented:
Ok, for my follow up on this.  I was able to resolve these issues.   I found an old hidden reference in a shadow protect article relating to Dynamic Cache and Windows that did affect Server 2008 R2 and prior in rare cases.    I followed the instructions linked from the article to create a Dynamic Cache service on the computer and edited the created registry entries to detail the amount of cache resources to utilize.  This, so far, seems to have resolved this issue.     I'll link the two articles related to this fix below.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Matt W.Author Commented:
No one else provided any usable help whatsoever. My solution is the one that worked.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.