Solved

VBA - Large StreamWriter Memory Exception

Posted on 2014-03-04
4
291 Views
Last Modified: 2014-03-04
Selecting data from a database, building StringBuilder. Trying to write a file using StreamWriter.

Using objWriter As New StreamWriter(strScriptNameMulti, False)
                Debug.Print(sbInsertStatement.Length)
                objWriter.WriteLine(sbInsertStatement.ToString)

End Using


Get error: Exception of type 'System.OutOfMemoryException' was thrown.

Debug.Print result: sbInsertStatement.Length = 95567264

I can split it into two files. One ends up being 60 MB and the other is 32 MB. But I'd rather have one file.

Is there a limit to StreamWriter size?
0
Comment
Question by:bkienzle
4 Comments
 
LVL 18

Assisted Solution

by:UnifiedIS
UnifiedIS earned 250 total points
ID: 39903593
I doubt it's a limitation of the streamwriter object, probably just too much for your memory to handle on the writeline statement.  

I would try building in a loop to write the  text in chunks.
Dim iChunkSize As Integer = 100000
Dim iNumberOfChunks As Integer
Dim sbInsertStatement As New System.Text.StringBuilder
iNumberOfChunks = Math.Ceiling(sbInsertStatement.Length / iChunkSize)
For x As Integer = 0 To iNumberOfChunks - 1
            objWriter.WriteLine(sbInsertStatement.ToString.Substring(x * iChunkSize, iChunkSize))

Next
0
 
LVL 83

Expert Comment

by:CodeCruiser
ID: 39903674
What happens if you remove

 Debug.Print(sbInsertStatement.Length)
0
 
LVL 40

Accepted Solution

by:
Jacques Bourgeois (James Burger) earned 250 total points
ID: 39903738
Since the whole String is already built, you might try to use a FileStream instead of a StreamWriter. It usually requires less memory.

I have also read in a few places, although I never tested it, that setting the buffer size yourself, by using one of the constructors that has a buffer size parameter, can prevent a copy in memory, so that might also help.
0
 

Author Closing Comment

by:bkienzle
ID: 39903940
Tried the FileStream. It blows up also.
Must be something in my environment.
Will give the Chunk approach a shot.
Thanks for the suggestions.
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

For those of you who don't follow the news, or just happen to live under rocks, Microsoft Research released a beta SDK (http://www.microsoft.com/en-us/download/details.aspx?id=27876) for the Xbox 360 Kinect. If you don't know what a Kinect is (http:…
Parsing a CSV file is a task that we are confronted with regularly, and although there are a vast number of means to do this, as a newbie, the field can be confusing and the tools can seem complex. A simple solution to parsing a customized CSV fi…
It is a freely distributed piece of software for such tasks as photo retouching, image composition and image authoring. It works on many operating systems, in many languages.
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now