Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 433
  • Last Modified:

Start reading a File from a Specific Row or Position

Basically I know how to loop through the file with StreamReader (i.e. sr.ReadLine) but what I want to do is start reading a row from a specific point in the file.

My routine basically reads a file into StreamReader, loops through the starting set of lines to get to my Starting Point, and then it reads the lines I want extracted to another file.

Here is some example code:
//iStart is declared as int32, line is a String
//oWrite is a class that creates the new file

Private Function openFile(ByVal fileName As String, ByVal getCount As Int32)
            Try
                Dim sr As StreamReader = New StreamReader(fileName)
                Dim sLines As New System.Text.StringBuilder
                Do While sCount <> iStart
                    line = sr.ReadLine
                    sCount += 1
                Loop
                sCount = 0
                Do While sCount <> getCount
                    line = sr.ReadLine
                    sLines.Append(line & vbCrLf)
                    sCount += 1
                Loop
                oWrite.WriteParse(sLines.ToString(), sfileName)
                sr.Close()
                sCount = 0
            Catch ex As Exception
                Throw ex
            End Try
        End Function

The worst situations occur when I am loading a file that has 2 million rows and I need something that started after the 500,000th row.  It takes a while to loop to the 500,000th position in the file, and when it finally gets there I am having memory problems slowing down my computer.  Any suggestions on how to jump to the position without reading each line?
0
deadferret
Asked:
deadferret
  • 4
  • 2
1 Solution
 
vigridCommented:
In order to know (for sure) the current line number you are reading, you must read the whole file from the beginning to the current point and count the CR/LF occurences.

Overloads Overrides Public Function Read(Char(), Integer, Integer) As Integer

You could read a file fragment to a predefined buffer, say in 16kb chunks and analyze them manually to find CR/LFs. That would solve your problem with memory.

HTH
0
 
vigridCommented:
ms-help://MS.MSDNQTR.2004JAN.1033/cpref/html/frlrfSystemIOStreamReaderClassReadTopic.htm

MSDN help link
0
 
deadferretAuthor Commented:
How would I go about doing this?  "You could read a file fragment to a predefined buffer, say in 16kb chunks and analyze them manually to find CR/LFs. That would solve your problem with memory."
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
deadferretAuthor Commented:
I also continue to get this message when trying to read for a Cr/Lf.

"Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection."

Do I have to give it the full length of the file?
0
 
vigridCommented:
Private Shared Sub Main(ByVal args As String())
      Dim fs As FileStream
      If (args.Length = 1) Then
            fs = New FileStream(args(0), FileMode.Open)
      Else
            Console.Write("Enter file name: ")
            Dim fileName As String = Console.ReadLine
            fs = New FileStream(fileName, FileMode.Open)
      End If
      Dim buffer As Byte() = New Byte(262144) {}
      Dim bufferSize As Integer = buffer.Length
      Dim offset As Integer = 0
      Dim readBytes As Integer = 0
      Dim lines As Integer = 0
      Do
            readBytes = fs.Read(buffer, 0, bufferSize)
            offset = (offset + readBytes)
            Console.Write("Read {0} bytes... ", offset)
            Dim counter As Integer = 0
            Do While (counter < readBytes)
                  lines = (lines + IIf((buffer(counter++) = 10), 1, 0))
            Loop
            Console.WriteLine("{0} lines found", lines)
      Loop ((readBytes = bufferSize) AndAlso (offset <> fs.Length))
      fs.Close
End Sub
0
 
vigridCommented:
Above code is a translation from C# made by .NET Reflector. Sorry, I'm not an expert in VB.NET, but as the .NET Framework is the same to all of the languages, I'm trying to help :)

The magic number 262144 is 256kbytes of memory - size of the buffer.
The magic number 10 is ASCII code for the newline character.

Above reads the file in 0.25MB chunks and searches for the newline characters. When you find enough of them, you must note the offset where it did occur (should be something like: seekOffset = offset - readBytes + counter) and use the FileStream.Seek method to point your stream at the beginning of the line you need.

HTH
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

  • 4
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now