Solved

I/O Reading data by chunks

Posted on 2007-11-28
6
152 Views
Last Modified: 2013-11-26
I'm using VS 2005, .net 2.0, and coding in VB.net.
I want to read in a text file that is 26MB in size.  Each record lengh is139 characters long.  Does someone know of a method to loop thru my file reading 139 characters at a time until I get to the end
of the file?

I explored using the StreamReader method but I'm getting the first 10K characters when I use
the ReadLine method.

Please let me know if you need additional information.

Thanks in advance.
0
Comment
Question by:GLlano
6 Comments
 
LVL 6

Accepted Solution

by:
-DJL- earned 250 total points
ID: 20370378
0
 
LVL 55

Assisted Solution

by:Jaime Olivares
Jaime Olivares earned 250 total points
ID: 20370405
Having a StreamReader sr:

String str as String
Dim buffer(139) as Char

sr.Read(c, 0, c.Length)
str = new String(buffer)
0
 
LVL 3

Expert Comment

by:wizrr
ID: 20370446
You need to read strings? Not binary data? If so you must know how file is encoded and use StreamReader to decode it. After that use Read instead of ReadLine.
Using sr As StreamReader = New StreamReader("TestFile.txt", Encoding.UTF8)

                Dim line As String

                ' Read and display the lines from the file until the end 

                ' of the file is reached.

                Dim buffer As Char[139];

                Dim str As StringBuilder

                Dim read As Int

                Do

                    

                    read = sr.Read(buffer, 0, 139);

                    str.Append(buffer)

                    Console.WriteLine(buffer)

                Loop Until Not read = 0

                sr.Close()

            End Using

Open in new window

0
 
LVL 16

Expert Comment

by:gnoon
ID: 20370522
Simply use TextFieldParser class of .NET 2.0.
See example here http://msdn2.microsoft.com/en-us/library/zezabash(VS.80).aspx
Here is code snippet & modified
Using Reader As New _

Microsoft.VisualBasic.FileIO.TextFieldParser("C:\TestFolder\test.log")

   Reader.TextFieldType = _

   Microsoft.VisualBasic.FileIO.FieldType.FixedWidth

   Reader.SetFieldWidths(139)

   Dim currentRow As String()

   While Not Reader.EndOfData

      Try

         currentRow = Reader.ReadFields()

         Dim currentField As String = currentRow(0)

         Console.WriteLine(currentField)

      Catch ex As Microsoft.VisualBasic.FileIO.MalformedLineException

         MsgBox("Line " & ex.Message & _

         "is not valid and will be skipped.")

      End Try

   End While

End Using

Open in new window

0
 
LVL 1

Expert Comment

by:Computer101
ID: 20615326
Forced accept.

Computer101
EE Admin
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This document covers how to connect to SQL Server and browse its contents.  It is meant for those new to Visual Studio and/or working with Microsoft SQL Server.  It is not a guide to building SQL Server database connections in your code.  This is mo…
The ECB site provides FX rates for major currencies since its inception in 1999 in the form of an XML feed. The files have the following format (reducted for brevity) (CODE) There are three files available HERE (http://www.ecb.europa.eu/stats/exch…
This Micro Tutorial hows how you can integrate  Mac OSX to a Windows Active Directory Domain. Apple has made it easy to allow users to bind their macs to a windows domain with relative ease. The following video show how to bind OSX Mavericks to …
Migrating to Microsoft Office 365 is becoming increasingly popular for organizations both large and small. If you have made the leap to Microsoft’s cloud platform, you know that you will need to create a corporate email signature for your Office 365…

920 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now