Solved

C# Programming How to grep columns/lines from Text File?

Posted on 2010-11-18
19
1,670 Views
Last Modified: 2013-12-14
Hi there! I have a C# console program which main functions should let a user grep lines / columns from a log text file.

An Example within the text file the user wishes to grep a group of all the related lines starting from a particular date etc. "Tue Aug 03 2004 22:58:34" to "Wed Aug 04 2004 00:56:48". Therefore after processing, the program would then output  all the data found within the log text files between the 2 dates.

Could someone please advise on some codes that I could use to grep or create a filter to retrieve the neccessary text/data from the file? Thanks!

C# Program Files:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.IO;

namespace Testing
{
    class Analysis
    {
        static void Main()
        {
            // Read the file lines into a string array.
            string[] lines = System.IO.File.ReadAllLines(@"C:\Test\ntfs.txt");

            System.Console.WriteLine("Analyzing ntfs.txt:");

            foreach (string line in lines)
            {
                Console.WriteLine("\t" + line);

                //  ***Trying to filter/grep out dates, file size, etc****
                if (lines = "Sun Nov 19 2000")
                {
                    Console.WriteLine("Print entire line");
                }
            }

            // Keep the console window open in debug mode.
            Console.WriteLine("Press any key to exit.");
            System.Console.ReadKey();
        }
    }
}

Open in new window


Log Text File Example:
Fri May 14 2004 09:56:58  1581056 m... r/rrwxrwxrwx 0        0        8168-128-3 C:/Program Files/AccessData/AccessData Forensic Toolkit/Program/IsoBuster.dll
Tue May 25 2004 14:34:12  5233391 m... r/rrwxrwxrwx 0        0        8355-128-3 C:/Program Files/AccessData/AccessData Forensic Toolkit/FTKManual.pdf
Mon Jun 07 2004 08:00:00        0 .a.. r/rrwxrwxrwx 0        0        8626-128-1 C:/RECYCLER/S-1-5-21-606747145-1004336348-839522115-1003/Dc1.LOG
Tue Jun 08 2004 03:50:55        0 ...b r/rrwxrwxrwx 0        0        8626-128-1 C:/RECYCLER/S-1-5-21-606747145-1004336348-839522115-1003/Dc1.LOG
Tue Jun 08 2004 03:50:56        0 m... r/rrwxrwxrwx 0        0        8626-128-1 C:/RECYCLER/S-1-5-21-606747145-1004336348-839522115-1003/Dc1.LOG
Fri Jul 16 2004 15:49:34   352256 m... r/rrwxrwxrwx 0        0        8358-128-3 C:/Program Files/AccessData/Common Files/AccessData LanguageSelector/LanguageSelector.exe
Sat Jul 17 2004 11:41:52   792644 m... r/rrwxrwxrwx 0        0        9098-128-3 C:/WINDOWS/system32/spool/drivers/w32x86/3/PSCRIPT.NTF
Wed Jul 21 2004 16:58:48   499712 m... r/rrwxrwxrwx 0        0        8360-128-3 C:/Program Files/AccessData/Common Files/AccessData LicenseManager/LicenseManager.exe
Tue Aug 03 2004 22:58:34    23040 m... r/rrwxrwxrwx 0        0        8522-128-3 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0002101.sys
                            23040 m... r/rrwxrwxrwx 0        0        9132-128-3 C:/WINDOWS/system32/ReinstallBackups/0003/DriverFiles/i386/mouclass.sys
                            23040 m... r/rrwxrwxrwx 0        0        9135-128-4 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0003123.sys
                            23040 m... r/rrwxrwxrwx 0        0        9136-128-3 C:/WINDOWS/system32/drivers/mouclass.sys
Tue Aug 03 2004 23:01:16   196864 m... r/rrwxrwxrwx 0        0        4706-128-3 C:/WINDOWS/system32/drivers/rdpdr.sys
Tue Aug 03 2004 23:08:18    24960 m... r/rrwxrwxrwx 0        0        8690-128-3 C:/WINDOWS/system32/drivers/hidparse.sys
Tue Aug 03 2004 23:08:20    36224 m... r/rrwxrwxrwx 0        0        8689-128-3 C:/WINDOWS/system32/drivers/hidclass.sys
Tue Aug 03 2004 23:08:38    20480 m... r/rrwxrwxrwx 0        0        7937-128-3 C:/WINDOWS/system32/drivers/usbuhci.sys
Tue Aug 03 2004 23:08:44    57600 m... r/rrwxrwxrwx 0        0        7935-128-3 C:/WINDOWS/system32/drivers/usbhub.sys
                           142976 m... r/rrwxrwxrwx 0        0        7936-128-3 C:/WINDOWS/system32/drivers/usbport.sys
Tue Aug 03 2004 23:08:48    26496 m... r/rrwxrwxrwx 0        0        7945-128-3 C:/WINDOWS/system32/drivers/USBSTOR.SYS
                            31616 m... r/rrwxrwxrwx 0        0        8687-128-3 C:/WINDOWS/system32/drivers/usbccgp.sys
Tue Aug 03 2004 23:14:38    52736 m... r/rrwxrwxrwx 0        0        8523-128-3 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0002102.sys
                            52736 m... r/rrwxrwxrwx 0        0        9131-128-3 C:/WINDOWS/system32/ReinstallBackups/0003/DriverFiles/i386/i8042prt.sys
                            52736 m... r/rrwxrwxrwx 0        0        9137-128-4 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0003122.sys
                            52736 m... r/rrwxrwxrwx 0        0        9138-128-3 C:/WINDOWS/system32/drivers/i8042prt.sys
Wed Aug 04 2004 00:56:46   464384 m... r/rrwxrwxrwx 0        0        9094-128-3 C:/WINDOWS/system32/spool/drivers/w32x86/3/PSCRIPT5.DLL
                           132608 m... r/rrwxrwxrwx 0        0        9095-128-3 C:/WINDOWS/system32/spool/drivers/w32x86/3/PS5UI.DLL
Wed Aug 04 2004 00:56:48    74240 m... r/rrwxrwxrwx 0        0        7934-128-3 C:/WINDOWS/system32/usbui.dll
Wed Aug 04 2004 01:01:08    40840 m... r/rrwxrwxrwx 0        0        4705-128-3 C:/WINDOWS/system32/drivers/termdd.sys
Wed Aug 04 2004 01:06:34     4821 m... r/rrwxrwxrwx 0        0        5318-128-3 C:/Program Files/Messenger/logowin.gif
                             7047 m... r/rrwxrwxrwx 0        0        5319-128-3 C:/Program Files/Messenger/lvback.gif
                            82944 m... r/rrwxrwxrwx 0        0        5320-128-3 C:/Program Files/Messenger/msgsc.dll
                           180224 m... r/rrwxrwxrwx 0        0        5321-128-3 C:/Program Files/Messenger/msgslang.dll
                          1667584 m... r/rrwxrwxrwx 0        0        5322-128-3 C:/Program Files/Messenger/msmsgs.exe
Wed Aug 04 2004 01:06:36     4454 m... r/rrwxrwxrwx 0        0        5323-128-3 C:/Program Files/Messenger/type.wav
                           115981 m... r/rrwxrwxrwx 0        0        5324-128-3 C:/Program Files/Messenger/xpmsgr.chm
Wed Aug 04 2004 06:58:30    60800 ...b r/rrwxrwxrwx 0        0        2344-128-3 C:/WINDOWS/system32/drivers/arp1394.sys
                            61824 ...b r/rrwxrwxrwx 0        0        2810-128-3 C:/WINDOWS/system32/drivers/nic1394.sys
Wed Aug 04 2004 06:58:34    23040 ...b r/rrwxrwxrwx 0        0        8522-128-3 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0002101.sys
                            23040 ...b r/rrwxrwxrwx 0        0        9136-128-3 C:/WINDOWS/system32/drivers/mouclass.sys
Wed Aug 04 2004 06:58:42     4352 ...b r/rrwxrwxrwx 0        0        2049-128-3 C:/WINDOWS/system32/drivers/swenum.sys
Wed Aug 04 2004 06:59:00  2056832 ...b r/rrwxrwxrwx 0        0        3477-128-3 C:/WINDOWS/system32/ntkrnlpa.exe

Open in new window

0
Comment
Question by:VMthinker
  • 11
  • 8
19 Comments
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 34165871
It is a bit unclear as to what your user input is. Is it a single begin date or is it a start and end date?
0
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 34165891
As for searching for the text, I believe string.StartsWith() will suit this situation:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.IO;

namespace Testing
{
    class Analysis
    {
        static void Main()
        {
            // Read the file lines into a string array.
            string[] lines = System.IO.File.ReadAllLines(@"C:\Test\ntfs.txt");

            System.Console.WriteLine("Analyzing ntfs.txt:");

            foreach (string line in lines)
            {
                Console.WriteLine("\t" + line);

                //  ***Trying to filter/grep out dates, file size, etc****
                if (line.StartsWith("Sun Nov 19 2000"))
                {
                    Console.WriteLine("Print entire line");
                }
            }

#if DEBUG
            // Keep the console window open in debug mode.
            Console.WriteLine("Press any key to exit.");
            System.Console.ReadKey();
#endif
        }
    }
}

Open in new window

0
 
LVL 2

Author Comment

by:VMthinker
ID: 34165985
Yea.... But what if some of the array has no start date on its lines array? This is due to multiple processes happening at the same time. Any ideas on that? Thanks.
0
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 34166016
What is the answer to, "Is it a single begin date or is it a start and end date?"
0
 
LVL 2

Author Comment

by:VMthinker
ID: 34166057
Sorry about that. It is a start date to end date thing. The program is more to like a timeline thing. Thanks.
0
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 34166073
One more question, inclusive or exclusive dates?
0
 
LVL 2

Author Comment

by:VMthinker
ID: 34166163
I am a bit confuse about the inclusive or exclusive dates question but I need the dates to be used as the main form of greping/filtering the results like a timeline of the various system processes during the start to end date. So I guess its inclusive?
0
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 34166330
Inclusive vs exclusive means if you say start date is "Tue Aug 03 2004", then dates that fall on or after "Tue Aug 03 2004" will be printed. Exclusive means dates that only fall after "Tue Aug 03 2004" will be printed.

I am curious about one thing: when I think of "grep", I think of passing in any string (or pattern in some versions) and getting a list of lines containing that string. Here, I get the feeling you are creating a more specific search filter in that you want to print all lines in a range, where the range is set by your users. Maybe grep does this, but I'm not terribly experienced with it. Am I missing something?
0
 
LVL 2

Author Comment

by:VMthinker
ID: 34169332
I am also not sure about where should the parameters be thus I have added the "If" statement. The grep function that was originally thought of was actually due to the scripting knowledge I usually utilize in Unix scripts but I am unsure if that would work in C#. Do you have any website of sample codes that I can utilize that I can use C# date range within a log text files or could you tweak the codes above abit? Thanks.
0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 74

Accepted Solution

by:
käµfm³d   👽 earned 500 total points
ID: 34172720
Here's what I'm thinking:  Take in a date range as two command line parameters (since you're thinking grep), validate the dates as valid, then return all text lines falling between and including those dates. Following is code describing this. I am passing dates in the form "mm/dd/yyyy" at the command line, but you should be able to use other formats as well. You can read up on the DateTime structure and its available methods to play around with how you receive your dates as parameters:  http://msdn.microsoft.com/en-us/library/system.datetime.aspx
class Program

{

    // Expected parameters:  1 - startDate; 2 - endDate; 3 - filename

    static void Main(string[] args)

    {

        if (args.Length >= 3)

        {

            DateTime startDate, endDate;



            if (DateTime.TryParse(args[0], out startDate) && DateTime.TryParse(args[1], out endDate))

            {

                if (System.IO.File.Exists(args[2]))

                {

                    string[] lines = System.IO.File.ReadAllLines(args[2]);

                    DateTime temp;

                    System.Globalization.CultureInfo ci = new System.Globalization.CultureInfo("en-US");

                    bool foundStartDate = false;



                    for (int i = 0; i < lines.Length; i++)

                    {

                        // DateTime.TryParseExact() allows you to pass a format string of the expected date.

                        //  It returns true if the value is a date or false otherwise

                        //

                        // "ddd MMM dd yyyy" this should match dates like "Tue Aug 03 2004"

                        if (DateTime.TryParseExact(lines[i].Substring(0, 15), "ddd MMM dd yyyy", ci, System.Globalization.DateTimeStyles.None, out temp))

                        {

                            if (foundStartDate)

                            {

                                if (temp >= endDate)

                                {

                                    break;

                                }

                                else

                                {

                                    Console.WriteLine(lines[i]);

                                }

                            }

                            else if (temp >= startDate)

                            {

                                foundStartDate = true;

                                Console.WriteLine(lines[i]);

                            }

                        }

                        else  // startDate has been found, but the date part may be blank, so we print this line

                              //  since it should be between start and end

                        {

                            if (foundStartDate)

                            {

                                Console.WriteLine(lines[i]);

                            }

                        }

                    }

                }

                else

                {

                    // Display invalid file and usage prompt

                }

            }

            else

            {

                // Display invalid date and usage prompt

            }

        }

        else

        {

            // Display usage prompt

        }

    }

}

Open in new window

0
 
LVL 2

Author Comment

by:VMthinker
ID: 34174501
Thanks for your reply kauf but do you think you can help me to tweak the codes you've shown me to time format instead?

I apologize for the extra work but I've managed to find a way to set a date limit to the original perl script needed to generate the logs. Now all thats left is to allow the user to select the time example "22:58:34" to "23:01:16" on the same day?

I really appriciate the help provided and hope to hear your reply.
0
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 34174678
As far as passing the time on the command line, you'll need to make sure your users know to quote it ( "02/04/2010 08:03:00" for example). This is a limitation of Windows (and others), not of C#. Other than that, the only modification you should need is at line 25. Change the format string from

    "ddd MMM dd yyyy"

to

    "ddd MMM dd yyyy hh:mm:ss"

and you should be set.
0
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 34174692
Note:

The "MMM" for month abbreviation and the "mm" for minutes are case-sensitive and should be entered as displayed.
0
 
LVL 2

Author Comment

by:VMthinker
ID: 34177997
Is it possible to only have the hh:mm:ss only as I think the script would be able to automatically filter out the date or do you recommend putting in the date as well? Thanks.
0
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 34179918
Have it where? When reading lines from the file or as input from the command line?
0
 
LVL 2

Author Comment

by:VMthinker
ID: 34180127
Thanks for your reply regarding your above question I guess that it would have to be both?

Just to ask is the codes provided suppose to run as an independent .exe program on the CMD? I tried to run it with these command on the CMD "Testing.exe Fri Nov 16 2000 Sat Aug 2003 C:\Test.ntfs"  with random dates not exact dates but it does not show any results except for a blank line. Is the error due to the spaces where the dates were placed in?

Or am I suppose yo use the class to be implemented inside my main program? If so where should I put the date parameters?

Once again thanks for your help.

0
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 34188963
This is the expected command line input:

    appname.exe "Fri Nov 16 2000" "Sat Aug 2003" "C:\Test.ntfs"

Note: You only need to quote command line parameters when they contain spaces. In the above example, "C:\Test.ntfs" does not have to be quoted since it has no spaces. Example:

    appname.exe "Fri Nov 16 2000" "Sat Aug 2003" C:\Test.ntfs
0
 
LVL 2

Author Closing Comment

by:VMthinker
ID: 34189061
Thanks kaufmed! Your codes and explination really helped!
0
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 34189154
Sure. Glad to help  :)
0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

Suggested Solutions

This article describes a simple method to resize a control at runtime.  It includes ready-to-use source code and a complete sample demonstration application.  We'll also talk about C# Extension Methods. Introduction In one of my applications…
It was really hard time for me to get the understanding of Delegates in C#. I went through many websites and articles but I found them very clumsy. After going through those sites, I noted down the points in a easy way so here I am sharing that unde…
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.
The viewer will learn how to synchronize PHP projects with a remote server in NetBeans IDE 8.0 for Windows.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now