Solved

Read a text file from bottom to top

Posted on 2011-02-16
4
960 Views
Last Modified: 2012-05-11
Hello Guys,

I am using Visual Basic .net 2008 and need a little help.

I have several text files which contain some data. The data in text files is stored in a format like -

date,time,value1,value2,value3
date,time,value1,value2,value3


Now each file is over 2 MB in size. I need to read file from last line and check if I need to replace a record or not. If yes, then I can read the entire file and do the needful. If not then goto another file.

Now I do not want to read file from start to end. This will waste time specially if I figure that I do not need to modify the file. So how can I read from bottom to top so that as soon I come across the record I want to check I can determine if I need to modify the file or not. The possibility of finding required record lower down is very high.

I want this operation to run as fast as possible.

If you guys can give me a small code thru which I can read file from last record onwards to top and as fast as possible it will be great.

Thank you,

Cheers,
Saurabh
0
Comment
Question by:Sukhani
4 Comments
 
LVL 53

Expert Comment

by:Dhaest
ID: 34904485
The only thing that I think of is either read the entire file and get the last line ...
http://www.tek-tips.com/viewthread.cfm?qid=1093173&page=1

Or using some kind of seek-mechanism
http://www.xtremedotnettalk.com/showthread.php?t=74284
0
 
LVL 18

Accepted Solution

by:
Dennis Aries earned 250 total points
ID: 34904492
You would need FileStream.Seek to do that.
Microsoft MSDN has an example that reads a file backwards byte for byte, but you can easily convert that to longer sections.
Dim bytes() As Byte = New Byte(lengthOfRecord - 1) {}
fs.Seek(0, , SeekOrigin.End) ' jump to end of file '
fs.Seek(-lengthOfRecord, SeekOrigin.Current) ' go back a single record '
fs.Read(bytes, 0, lengthOfRecord) ' read your record into the byte-array '

Open in new window

0
 
LVL 10

Assisted Solution

by:John Claes
John Claes earned 250 total points
ID: 34904529


I can see 2 possiblilities. The first is my solution to this.
1) to read the Lines with the posiblity to use Linq to retrieve 1 line Fast : System.IO.File.ReadLines
http://msdn.microsoft.com/en-us/library/dd383503.aspx


2) to use the Filestream offset and Count to do you're own searching
http://msdn.microsoft.com/en-us/library/system.io.filestream.read.aspx


This is an Example of ReadLines upon the IO.File object

The following example uses the ReadLines method in a LINQ query that enumerates all directories for files that have a .txt extension, reads each line of the file, and displays the line if it contains the string "Microsoft".


regards
poor beggar
Imports System.IO
Imports System.Xml.Linq
Module Module1

    Sub Main()

        Try
            Dim files = From chkFile In Directory.EnumerateFiles("c:\", "*.txt", _
                                                 SearchOption.AllDirectories)
                        From line In File.ReadLines(chkFile)
                        Where line.Contains("Microsoft")
                        Select New With {.curFile = chkFile, .curLine = line}

            For Each f In files
                Console.WriteLine("{0}\t{1}", f.curFile, f.curLine)
            Next
			Console.WriteLine("{0} files found.", _
					files.Count.ToString())
        Catch UAEx As UnauthorizedAccessException
            Console.WriteLine(UAEx.Message)
        Catch PathEx As PathTooLongException
            Console.WriteLine(PathEx.Message)
        End Try
    End Sub
End Module

Open in new window

0
 
LVL 4

Expert Comment

by:asfahani
ID: 34904633
This code reads your file from the buttom to the top. Where i commented is the place where you need to do your testing. this code is 7x faster than using the readline method.

Public Sub ReadTextFileBackwards(ByVal sFilePath As String, ByVal sSearchString As String)
    Dim i As Integer

    Dim streamTextFile As Stream

    streamTextFile = File.OpenRead(sFilePath)
    streamTextFile.Seek(0, SeekOrigin.End)

    Dim stringArray() As String
    Dim sBuffer As String = ""

    Dim iBlockSize As Integer = 41000
    Dim iFirstElement As Integer = 1

    While streamTextFile.Position > 0
      If streamTextFile.Position <= iBlockSize Then
        iBlockSize = CInt(streamTextFile.Position)
        iFirstElement = 0
      End If
      Dim byteArray(iBlockSize - 1) As Byte
      streamTextFile.Seek(-1 * iBlockSize, SeekOrigin.Current)
      streamTextFile.Read(byteArray, 0, byteArray.Length)
      streamTextFile.Seek(-1 * iBlockSize, SeekOrigin.Current)
      stringArray = Split(ASCII.GetString(byteArray), vbCrLf)
      stringArray(stringArray.Length - 1) = stringArray(stringArray.Length - 1) + sBuffer
      For i = stringArray.GetUpperBound(0) To iFirstElement Step -1
        'Do Your Check If The Line has something that requires your file to be changed
        ' eg: if stringArray(i) = "123456" do Bla Bla Bla
      Next
      sBuffer = stringArray(0)
    End While

  End Sub
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Latency in .net app using DB in .net 21 65
VB.NET 1 23
Batch file or script with password 22 44
Help with preventing downloading a zip file 10 37
In my previous two articles we discussed Binary Serialization (http://www.experts-exchange.com/A_4362.html) and XML Serialization (http://www.experts-exchange.com/A_4425.html). In this article we will try to know more about SOAP (Simple Object Acces…
The ECB site provides FX rates for major currencies since its inception in 1999 in the form of an XML feed. The files have the following format (reducted for brevity) (CODE) There are three files available HERE (http://www.ecb.europa.eu/stats/exch…
This Micro Tutorial hows how you can integrate  Mac OSX to a Windows Active Directory Domain. Apple has made it easy to allow users to bind their macs to a windows domain with relative ease. The following video show how to bind OSX Mavericks to …
Microsoft Active Directory, the widely used IT infrastructure, is known for its high risk of credential theft. The best way to test your Active Directory’s vulnerabilities to pass-the-ticket, pass-the-hash, privilege escalation, and malware attacks …

809 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question