Solved

VBA - Large StreamWriter Memory Exception

Posted on 2014-03-04
4
293 Views
Last Modified: 2014-03-04
Selecting data from a database, building StringBuilder. Trying to write a file using StreamWriter.

Using objWriter As New StreamWriter(strScriptNameMulti, False)
                Debug.Print(sbInsertStatement.Length)
                objWriter.WriteLine(sbInsertStatement.ToString)

End Using


Get error: Exception of type 'System.OutOfMemoryException' was thrown.

Debug.Print result: sbInsertStatement.Length = 95567264

I can split it into two files. One ends up being 60 MB and the other is 32 MB. But I'd rather have one file.

Is there a limit to StreamWriter size?
0
Comment
Question by:bkienzle
4 Comments
 
LVL 18

Assisted Solution

by:UnifiedIS
UnifiedIS earned 250 total points
ID: 39903593
I doubt it's a limitation of the streamwriter object, probably just too much for your memory to handle on the writeline statement.  

I would try building in a loop to write the  text in chunks.
Dim iChunkSize As Integer = 100000
Dim iNumberOfChunks As Integer
Dim sbInsertStatement As New System.Text.StringBuilder
iNumberOfChunks = Math.Ceiling(sbInsertStatement.Length / iChunkSize)
For x As Integer = 0 To iNumberOfChunks - 1
            objWriter.WriteLine(sbInsertStatement.ToString.Substring(x * iChunkSize, iChunkSize))

Next
0
 
LVL 83

Expert Comment

by:CodeCruiser
ID: 39903674
What happens if you remove

 Debug.Print(sbInsertStatement.Length)
0
 
LVL 40

Accepted Solution

by:
Jacques Bourgeois (James Burger) earned 250 total points
ID: 39903738
Since the whole String is already built, you might try to use a FileStream instead of a StreamWriter. It usually requires less memory.

I have also read in a few places, although I never tested it, that setting the buffer size yourself, by using one of the constructors that has a buffer size parameter, can prevent a copy in memory, so that might also help.
0
 

Author Closing Comment

by:bkienzle
ID: 39903940
Tried the FileStream. It blows up also.
Must be something in my environment.
Will give the Chunk approach a shot.
Thanks for the suggestions.
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

In my previous article (http://www.experts-exchange.com/Programming/Languages/.NET/.NET_Framework_3.x/A_4362-Serialization-in-NET-1.html) we saw the basics of serialization and how types/objects can be serialized to Binary format. In this blog we wi…
This article describes relatively difficult and non-obvious issues that are likely to arise when creating COM class in Visual Studio and deploying it by professional MSI-authoring tools. It is assumed that the reader is already familiar with the cla…
This Micro Tutorial will teach you how to censor certain areas of your screen. The example in this video will show a little boy's face being blurred. This will be demonstrated using Adobe Premiere Pro CS6.
This Micro Tutorial demonstrates using Microsoft Excel pivot tables, how to reverse engineer competitors' marketing strategies through backlinks.

803 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question