SSIS - "Not enough storage is available to complete this operation" when executing Directory.GetFiles method in a ScriptTask

Posted on 2013-11-07
Last Modified: 2016-02-11
My environment:

- Visual Studio 2008 SP1

- Sql Server 2008 x64

- Developement machine: Windows 8.1 x64 all updates installed - i7quad - 8GB Memory

- Sql Server production machine: Cluster (A/P): 2x Windows 2008 R2 x64 - 32GB Memory

This problem is driving me crazy.
Some production SSIS jobs access an AS400 folder (QDLS) for their processing.
In particular, they call the 7zip executable to create an archive with the files in a QDLS sub-folder and send it via ftp.
No problem so far. Everything works correctly.

After processing is complete (after the archive has been sent), the job should move the processed files to a backup folder.
So, within a "Script Task" I put a C# code that deals with this.

But after many months of executions correct, the jobs are now in error at the point: Directory.GetFiles("path_to_qdls");

I can replicate the error by debugging the script from my pc:

Error: 0x1 at Script Task: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Exception: \\192.168.x.x\qdls\ ---> System.IO.IOException: Not enough storage is available to complete this operation.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Directory.InternalGetFileDirectoryNames(String path, String userPathOriginal, String searchPattern, Boolean includeFiles, Boolean includeDirs, SearchOption searchOption)
   at System.IO.Directory.GetFiles(String path, String searchPattern, SearchOption searchOption)
   at System.IO.Directory.GetFiles(String path)
   at ST_76f7dac7d52b44b6b988a5c517655f1d.csproj.ScriptMain.Main() in C:\Users\x\AppData\Local\Temp\SSIS\1792c6065afc40ba883b89b83583cb85\ScriptMain.cs:line 61
   --- End of inner exception stack trace ---
   at ST_76f7dac7d52b44b6b988a5c517655f1d.csproj.ScriptMain.Main() in C:\Users\x\AppData\Local\Temp\SSIS\1792c6065afc40ba883b89b83583cb85\ScriptMain.cs:line 80
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
   at System.RuntimeMethodHandle.InvokeMethodFast(Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)
   at System.Type.InvokeMember(String name, BindingFlags invokeAttr, Binder binder, Object target, Object[] args, CultureInfo culture)
   at Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTATaskScriptingEngine.ExecuteScript()
Task failed: Script Task
Warning: 0x80019002 at TEST: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED.  The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "TEST.dtsx" finished: Failure.
The program '[7548] TEST.dtsx: DTS' has exited with code 0 (0x0).

Does anybody have an idea of what is the cause of this strange behavior (strange to me)?

Please note that:

- Any folder I tried, other than QDLS, on the AS400 works fine.

- QDLS folder (even if it is not really a "windows share") has full permission for all users (something equivalent to "everyone - full control)

- The packages are executed in production with an ad-hoc user (named "SqlAg"), that is used for all the jobs.

- The problem can be replicated by debugging the package in VS2008, so it seems to me not to be a problem related to the executing user.

- The package executes a 7zip command-line, which takes files from the QDLS folder. This operation works without problems.

- It seems the problem is related to the directory listing only: I created a test dtsx with only a ScriptTask inside, and the script with only the Directory.GetFiles() line....

I have already tried to:

- Change Directory.GetFiles() with DirectoryInfo.GetFiles()

- Use the "dir" command wrapped into a method, instead of the above two lines.

- Change the executing user of the jobs, by creating a proxy to another (more globally privileged) user.
Question by:jonnidip
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 7
  • 5
LVL 35

Expert Comment

by:Gary Patterson
ID: 39630456
This error is thrown when an attempt to allocate memory in .NET fails.

GetFiles returns an array of String. This is probably occurring when allocating memory for the results array.

Maybe GetFiles is returning a very large (larger than in the past) result set.  How may files are there in QDLS?  Has that changed recently (or just grown continuously over time) where now there are a lot more files than in the past?

- Gary Patterson
LVL 13

Author Comment

ID: 39630475
In this particular case the qdls sub-folder is empty.
These folders I am working on usually contain from 0 to 10 files, not more.

Thank you for your help.
LVL 35

Expert Comment

by:Gary Patterson
ID: 39630591
If volumes of files are small, then it may be that some specialized resource is exhausted, like file handles.

Here's a good troubleshooter:

- Gary Patterson
PeopleSoft Has Never Been Easier

PeopleSoft Adoption Made Smooth & Simple!

On-The-Job Training Is made Intuitive & Easy With WalkMe's On-Screen Guidance Tool.  Claim Your Free WalkMe Account Now

LVL 13

Author Comment

ID: 39631072
Thank you for pointing me to that article. I have already read that, but I think I am in a different situation than the ones shown.
That article shows different solutions for logon-related problems. But if it was a logon issue, then why does it fail with my user too?
At the beginning of the article it is said that the problem seemed to be solved when the machine has been rebooted. Well, I switched the active cluster note to the other server, but it did not solve my problem (the other node has been rebooted previously).

Then, I think that if it was a problem of handles, then I should have seen an abnormal memory consumption, but it did not happen. In addition, the error occurs immediately when hitting the "GetFiles" method. There is no "filling time".
Then, I think that a file-handle-exausted related error may appear when the machine (or the process) has been running for a while, but I tried with a fresh-boot of the server and I tried locally on my PC.

I don't want to lead you to a possible bad way, but my opinion is that this could be a qdls problem. Qdls seems to be a special kind of folder in AS400 and I really don't know anything about it. I asked my AS400 colleague if it would be possible to save the files to another folder, but it seems to be quite difficult at the moment.

Any idea?
LVL 35

Accepted Solution

Gary Patterson earned 500 total points
ID: 39631591
I've been troubleshooting AS/400-to-Windows problems since about 1990, and AS/400-to -DOS problems before that.  This isn't exactly unfamiliar territory.

This is a Windows .NET method call that is failing and throwing an error.  

It is possible that it is something that went wrong on AS/400 and caused it?  Maybe, but not likely at the volumes you described.

What has changed recently?  Did they upgrade or install PTFs on the AS/400?  Did anything change on the Windows side?

Directory.GetFiles, when used against a remote system, ends up causing a SMB/CIFS transaction to be initiated.  A connection is established, authentication is performed, and messages are exchanged.  

Ultimately, a CIFS message is sent to request directory information from the remote system (the AS/400 in this case).  

On the AS/400, Windows file sharing requests are received and handled by a service called "NetServer".

Once connected and authenticated, NetServer will then request a list of files from the QDLS file system using the appropriate AS/400 API, and formats and sends the list back to the Windows requester - again using SMB/CIFS protocol messages.

The Windows box, according to you, is executing Directory.GetFiles() (a .NET method), and failing with a the .NET exception "System.IO.IOException: Not enough storage is available to complete this operation".

That happens when GetFiles() attempts to allocate Windows resources - and as you can see, there is a lot going on in Windows - connection allocation, authentication, redirection, communications - plenty of places for something to go wrong.

System.IO.IOException: Not enough storage is available to complete this operation is not an AS/400 message, and it isn't a SMB/CIFS message, as far as I know:

I just want to eliminate a runaway program on Windows before we start doing more complex troubleshooting.

QDLS is "special" in that it is designed for compatibility with DOS naming conventions - ("eight dot three").  It is old, and relatively slow, and has strict file naming restrictions.  It won't hurt anything to try to switch over to a different file system on the AS/400.  That should be very easy.  If your admin or developer doesn't know how, I'd be happy to explain.

I don't think you necessarily have a "login problem".

I do think you have some sort of resource allocation issue in Windows that is preventing GetFiles() from working properly, and I think you need to solve that problem.

As far as the "Not Enough Storage" article goes:  

This is a general purpose troubleshooter for anything that results in a "Not enough storage" error.  Not just for login issues.

So please check your handle counts when the error happens, and post what you see here.  

Open task manager
Go to the Process tab (show processes from  other users)
Add the "Handles" column.
Sort by Handles from big to little.

- Gary Patterson
LVL 13

Author Comment

ID: 39631886
Thank you Gary. Your comment is very professional.
I will try to investigate more, basing on your suggestions.

Thank you.
LVL 27

Expert Comment

ID: 39633347
- Any folder I tried, other than QDLS, on the AS400 works fine.

/QDLS really shouldn't be used by anything later then Windows 2000. If using one of the more modern file systems works, you'll save yourself future problems by switching. The /QDLS file system was last updated to be compatible with OS/2 and with Windows 2000.

LVL 35

Expert Comment

by:Gary Patterson
ID: 39633448
Just went back and re-read question from the top.

- Any folder I tried, other than QDLS, on the AS400 works fine.

- QDLS folder (even if it is not really a "windows share") has full permission for all users (something equivalent to "everyone - full control)

I think I misunderstood part of this question.

If you want to access QDLS remotely using "//ip.address/qdls/", then qdls MUST be shared in NetServer.  That is a basic pre-requisite.  NetServer doesn't "know" about a file system or folder unless it is shared (or if it is a child folder under a share).

Alternately, if there is a root share set up, then you could use "//ip.address/root_share_name/qdls"

But this will never work if QDLS isn't either shared directly, or found other another share.

- Gary
LVL 35

Expert Comment

by:Gary Patterson
ID: 39633462
Sharing QDLS is easy.  Here is the link to the IBM Info Center:

- Gary
LVL 13

Author Comment

ID: 39633478
The qdls folder IS somewhat "shared", I can access it with \\192.168.x.x\qdls.
Please note that directory listing from Windows Explorer or .NET "Directory.GetFiles()" or DOS "dir" works fine.
The problem exists only inside Script Task of SSIS jobs...

I still haven't tried all the suggestions above, because I am busy with other works.

LVL 35

Expert Comment

by:Gary Patterson
ID: 39633886
No such thing as "sort of shared".  There is either a share configured, or not.  If it is, it is accessible by authorized users if NetServer is running.  

If you can access it via \\192.168.x.x\qdls then it is shared.

What version of Windows and what AS/400 OS version are you using?

- Gary
LVL 13

Author Closing Comment

ID: 39677906
Gary, yours is a very very complete answer.
I tried your suggestions and ended up switching to a different folder (not qdls) on the AS400 side.

Thank you.
LVL 35

Expert Comment

by:Gary Patterson
ID: 39677919
Thanks for following up.  

Never tried to use QDLS from SSIS - I'm curious why there is an issue.  I may give it a shot on our dev system the next time I have to test something in SSIS.

Glad you got it worked out.

- Gary

Featured Post

Three Reasons Why Backup is Strategic

Backup is strategic to your business because your data is strategic to your business. Without backup, your business will fail. This white paper explains why it is vital for you to design and immediately execute a backup strategy to protect 100 percent of your data.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
Performance in games development is paramount: every microsecond counts to be able to do everything in less than 33ms (aiming at 16ms). C# foreach statement is one of the worst performance killers, and here I explain why.
In this video, viewers are given an introduction to using the Windows 10 Snipping Tool, how to quickly locate it when it's needed and also how make it always available with a single click of a mouse button, by pinning it to the Desktop Task Bar. Int…
NetCrunch network monitor is a highly extensive platform for network monitoring and alert generation. In this video you'll see a live demo of NetCrunch with most notable features explained in a walk-through manner. You'll also get to know the philos…

617 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question