"Insufficient system resources" errors while copying large Server 2003 files

I keep received the error message "Insufficient system resources" errors while copying large backup files on a Server 2003 server.

The server is a Windows Server 2003 server that has 4 GB of RAM. The paging file is set to an initial and a maximum size of 4096 MB.

The server is able to copy smaller and medium sized files without any problems.

However, when I try to copy larger files (25-90 GB in size) I always get these "Insufficient system resources" errors if I use the Windows Explorer copy and paste, cut and paste, or the command line copy or xcopy commands.

While I have been able to find a solution for copying these large backup files using the Microsoft Sync Toy program, I would like to fix the underlying server memory and system resources issues that are causing these Windows copies to fail.

The administrator who regularly administers this server needs to be able to do the regular Windows copies; he can't be expected to always use the Microsoft Sync Toy program.

So, what do I need to do to fix the underlying operating system memory and system resources issues that are causing these copy commands to fail with the "Insufficient system resources" errors?
IT GuyNetwork EngineerAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

I am not sure you can.  Perhaps installing teracopy, which can be set to kick in automatically, would tackle this issue?
IT GuyNetwork EngineerAuthor Commented:
Does anyone else know if there are any solutions, fixes, patches, or Windows updates that will help fix this issue?
The deal is, you can try increasing the page pool memory in the registry, which I believe is the MS party line on this,  but it seems that more than half the time this fails and its a waste of time.

Employing robocopy rarely gets around the problem.

Swapping out the memory could do it.  Defective mem, of memory not well matched to the Intel or AMD architecture can affect this issue.

I.e., you can search around for the page pool fix and try it.  You can then try using Robocopy.  You can then try different brands of memory... if that's really what you feel you need to do.

But the bottomline is, imho, that this is simply a shortcoming in the MS Operating system... it is intolerant of some quirky hardware limitations that are naturally going to exist depending on how suitable the mate is between your mobo architecture and memory.  Amd seems more viable than Intell in this regard (spotting rumors in the wild).

Personally, I wouldn't bother messing with page pool and memory swap until I see that teracopy won't handle it.

Just my personal ;^)

SolarWinds® IP Control Bundle (IPCB)

Combines SolarWinds IP Address Manager and User Device Tracker to help detect IP conflicts, quickly identify affected systems, and help your team take near instantaneous action. Help improve visibility and enhance reliability with SolarWinds IP Control Bundle.

Here's the info from MS

If it were me, I would first test with Robocopy, and then with Teracopy, to see if either or both of those fails before making page pool management changes, just so I have that info handy.  
After adjusting the page pool I would test -native, -robo, and -tera again.

If native problems persist but teracopy works, I might just leave it there, the next option seemingly being to try different brands of memory in the hopes you will overcome a hardware mismatch factor of 'slight-nuance'.

..i.e., its not so much that it is broken as the OS can't manage the difference-nuance.

But if you gotta have perfection, I think its going to come down to the memory, and or g-d-only-knows-what-other not totaly perfect hardware match that is throwing MS a curve.

...then again, maybe someone else has better insight into this matter than I (a non-expert ;^)
IT GuyNetwork EngineerAuthor Commented:
So Microsoft doesn't have any sort of "fixes" to help resolve this sort of issue?

As a network support person I am always quick to realize that some issues simply can't be easily fixed (if they can be fixed at all). I'm always willing to use third party programs like Teracopy, Microsoft Sync Toy, Robocopy, etc. as a work around to copy large files whenever I encounter these sorts of "insufficient system resources" error messages.

However, I have a very picky client who I support who keeps pushing me to find a solution for this insufficient system resources issue that occurs while he tries to copy these large Windows backup files. He wants me to be able to determine what the root cause of this issue is and to give him a recommendation on exactly what needs to be done to fix it. He doesn't want to have to rely on these sorts of third party programs or work abounds to resolve issues like this.

I'm simply trying to find out if there are any ways at all of resolving this issue that he has been having.
No, I know of no fixes from MS.
MS is known for its shortcomings in robust copy, and its been slow to evolve.

That doesn't mean there isn't some other cause for the problem, i.e., the one I addressed is likely the 'deal' once you have exhausted all obvious (and less-than-obvious) causes, like
 - hardware integrity: deep check of drives and memory and mobo (inlcuding cables and connectors)
 - software integrity: intimate observation of the backup creation (including logs and system logs, etc), and don't forget sfc for windows integrity

Have you run extended copies yourself while intimately monitoring resources ?

Check things like
 - how's the health of the registry
 - is the $mft healthy?  (checkdsk /r AND THEN --> testdisk)
 - hows the disk(s) mbr's
 - are there some files and/or pathways that are over-extended (character-wise) {usuallly this problem would become apparent before (ouside of) a backup - but you could look at that
 - http://support.microsoft.com/kb/320081  cause 4)

So there's possibly more of the usual suspects you could stir up with some pro-tools in hand.
Once you have done due diligence to the usual suspects, then I am looking at my second post as the next bridge, which means...

If your client is still hell bent on you "fixing" it at the point you have ruled out non-native weaknesses or issues, then its time to invest in change of hardware, but you must make it clear there are no guarantees, either that it will be solved for you, OR that if it is seemingly solved that it will not magically crop up again (over the course of windows updates), and its going to cost probably more than its worth.

It is for this reason, if I had an otherwise proven good (rockin') server, I would explain to the client why, in my expert opinion (after I threw my toolset at it, of course), a third party professional copy tool" is the correct answer (and commonly used to overcome limitations of native MS copy capability) ...unless he wants to spend lots of money for the satisfaction of maybe seeing it work perfectly native for a while.

Imo, It comes down to due diligence to make sure you are, indeed, running up against a natural limitation of the employed tools (os, apps, hardware).  

...see how we go with that...

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Another culprit in this that I have personally seen is MS SQL. MS SQL is designed to reserve any memory it sees as available. MS says that this is the intended behavior and the solution is to have SQL on a dedicated server. If your client is running SQL on this server, that could be a large contributor to the issue. In my example, it was causing an issue with backups rather than file copy and still the MS answer was essentially, "working as intended" because of SQL.
Thank you for the acceptance, Knowledgeable.
I hope this was helpful to you.
IT GuyNetwork EngineerAuthor Commented:

Thank you once again for all your help.

Some customers never will accept that fact that computers and Microsoft products have limitations and shortcomings.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Server 2003

From novice to tech pro — start learning today.