Stream Reader out of memory exception

I am using StreamReader to read in a text file in 2005
My problem is at the.last line 1 text file is malformed
it does not have a LFCR
so it keeps reading

I end up with 150 Megs read in and then it throws this error

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)
   at System.Text.StringBuilder.GetNewString(String currentString, Int32 requiredLength)
   at System.Text.StringBuilder.Append(Char[] value, Int32 startIndex, Int32 charCount)
   at System.IO.StreamReader.ReadLine()

My lines are at most 164 characters not counting the LFCR

I need a way to tell stream reader to stop reading after a 165 characters
any idea

Dim sr As StreamReader = File.OpenText(filename)
Do Until sr.Peek = -1
linecontent = sr.ReadLine
If Trim(linecontent) <> "" Then

If Trim(Mid(linecontent, 79, 8)) <> "" Then
aDate1 = Mid(linecontent, 79, 8)
aDate1 = Mid(aDate1, 5, 2) & "/" & Mid(aDate1, 7, 2) & "/" & Mid(aDate1, 1, 4)
aDate1 = "12/31/1849"
 End If

If CDate(aDate1) >= dDate
sGuest = Mid(linecontent, 1, 6) & "0"
End IF

Other code here that works fine

End IF
linecontent = String.Empty

Open in new window

Who is Participating?
do you need to use a binary reader?
Try this method:

Using sr As New StreamReader(<path>)
  While Not sr.EndOfStream
     'your code here for parsing
  End While
End Using 'this closes and disposes the stream

Open in new window

PS be sure to close your streams.
hainansyndromeAuthor Commented:
how do I setup the Stream Reader part?
I am trying

            Dim sr As StreamReader
            Using sr As New StreamReader=File.OpenText((destination & "\" & filename)

but i get end of Statement expected?
Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

hainansyndromeAuthor Commented:
sing sr As New StreamReader=File.OpenText((destination & "\" & filename)

is not working
hainansyndromeAuthor Commented:
ok never mind i have dain bramage

but that did not help
it still hit the last line and blew up

see the last line is about 170 megs since there are no LFCR
Mike TomlinsonMiddle School Assistant TeacherCommented:
May we see all the code?...
hainansyndromeAuthor Commented:
I have shown all the relevant code
the problem is on the last line of the text file
Mike TomlinsonMiddle School Assistant TeacherCommented:
You've shown all the code that YOU think is relevant...but you were unable to solve the problem.  If you show us all the code we may spot a problem that you are unaware of.  Just saying...
hainansyndromeAuthor Commented:
Idle_Mind: I have show you all the relevant code
I know with 10000000% certainty the remaining code has no flaws
I have trappend this error
I know when and where it is occuring
Mike TomlinsonMiddle School Assistant TeacherCommented:
The "OutOfMemoryException" part throws up a red flag.  It indicates that you were low on memory, which shouldn't be the case if you are only reading LINE BY LINE and each line is no longer than 164 chars.

Are you by chance, in that code you are sure is not the problem, building up a huge value with simple string concatenation?

    str = str & someDataExtractedFromEachLine

Or in short hand:

    str &= someDataExtractedFromEachLine
Why is the last line 170 megs? Maybe you need to rethink the file design.
Mike TomlinsonMiddle School Assistant TeacherCommented:
If you need to control how much is read at a time then use a straight FileStream() and then you can read by a specified number of bytes:

...but then you lose the built-in ability to read only the next line.  You have to manually look for the CRLF 's in the returned array of bytes and break up the data yourself.
hainansyndromeAuthor Commented:
I fixed it
must have been the fact that I had 8 VS windows open or that I have not rebooted in 4 days
after reboot the  .substring showed up

I was able to use
Dim sLen As Integer = 160
linecontent = sr.ReadLine.Substring(0, sLen)
I then later incremented up to 164 characters using a Try Catch
Mike TomlinsonMiddle School Assistant TeacherCommented:
Just for clarification, though, if the last line really is ~super~ long...your method directly above is NOT reading just 160 characters.  It is still reading the ENTIRE line and then Substring() is simply returning the requested number of chars.  This apparently is only working because you have enough memory available.  To truly read only the limited number of chars you'd have to use the lower level FileStream.Read() method as I mentioned earlier, or you something similar like BinaryReader() as deighton suggests.
hainansyndromeAuthor Commented:
I stand corrected
same problem
it hits the last line and bam

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.