Solved

C# Programming How to grep columns/lines from Text File?

Posted on 2010-11-18
19
1,705 Views
Last Modified: 2013-12-14
Hi there! I have a C# console program which main functions should let a user grep lines / columns from a log text file.

An Example within the text file the user wishes to grep a group of all the related lines starting from a particular date etc. "Tue Aug 03 2004 22:58:34" to "Wed Aug 04 2004 00:56:48". Therefore after processing, the program would then output  all the data found within the log text files between the 2 dates.

Could someone please advise on some codes that I could use to grep or create a filter to retrieve the neccessary text/data from the file? Thanks!

C# Program Files:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.IO;

namespace Testing
{
    class Analysis
    {
        static void Main()
        {
            // Read the file lines into a string array.
            string[] lines = System.IO.File.ReadAllLines(@"C:\Test\ntfs.txt");

            System.Console.WriteLine("Analyzing ntfs.txt:");

            foreach (string line in lines)
            {
                Console.WriteLine("\t" + line);

                //  ***Trying to filter/grep out dates, file size, etc****
                if (lines = "Sun Nov 19 2000")
                {
                    Console.WriteLine("Print entire line");
                }
            }

            // Keep the console window open in debug mode.
            Console.WriteLine("Press any key to exit.");
            System.Console.ReadKey();
        }
    }
}

Open in new window


Log Text File Example:
Fri May 14 2004 09:56:58  1581056 m... r/rrwxrwxrwx 0        0        8168-128-3 C:/Program Files/AccessData/AccessData Forensic Toolkit/Program/IsoBuster.dll
Tue May 25 2004 14:34:12  5233391 m... r/rrwxrwxrwx 0        0        8355-128-3 C:/Program Files/AccessData/AccessData Forensic Toolkit/FTKManual.pdf
Mon Jun 07 2004 08:00:00        0 .a.. r/rrwxrwxrwx 0        0        8626-128-1 C:/RECYCLER/S-1-5-21-606747145-1004336348-839522115-1003/Dc1.LOG
Tue Jun 08 2004 03:50:55        0 ...b r/rrwxrwxrwx 0        0        8626-128-1 C:/RECYCLER/S-1-5-21-606747145-1004336348-839522115-1003/Dc1.LOG
Tue Jun 08 2004 03:50:56        0 m... r/rrwxrwxrwx 0        0        8626-128-1 C:/RECYCLER/S-1-5-21-606747145-1004336348-839522115-1003/Dc1.LOG
Fri Jul 16 2004 15:49:34   352256 m... r/rrwxrwxrwx 0        0        8358-128-3 C:/Program Files/AccessData/Common Files/AccessData LanguageSelector/LanguageSelector.exe
Sat Jul 17 2004 11:41:52   792644 m... r/rrwxrwxrwx 0        0        9098-128-3 C:/WINDOWS/system32/spool/drivers/w32x86/3/PSCRIPT.NTF
Wed Jul 21 2004 16:58:48   499712 m... r/rrwxrwxrwx 0        0        8360-128-3 C:/Program Files/AccessData/Common Files/AccessData LicenseManager/LicenseManager.exe
Tue Aug 03 2004 22:58:34    23040 m... r/rrwxrwxrwx 0        0        8522-128-3 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0002101.sys
                            23040 m... r/rrwxrwxrwx 0        0        9132-128-3 C:/WINDOWS/system32/ReinstallBackups/0003/DriverFiles/i386/mouclass.sys
                            23040 m... r/rrwxrwxrwx 0        0        9135-128-4 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0003123.sys
                            23040 m... r/rrwxrwxrwx 0        0        9136-128-3 C:/WINDOWS/system32/drivers/mouclass.sys
Tue Aug 03 2004 23:01:16   196864 m... r/rrwxrwxrwx 0        0        4706-128-3 C:/WINDOWS/system32/drivers/rdpdr.sys
Tue Aug 03 2004 23:08:18    24960 m... r/rrwxrwxrwx 0        0        8690-128-3 C:/WINDOWS/system32/drivers/hidparse.sys
Tue Aug 03 2004 23:08:20    36224 m... r/rrwxrwxrwx 0        0        8689-128-3 C:/WINDOWS/system32/drivers/hidclass.sys
Tue Aug 03 2004 23:08:38    20480 m... r/rrwxrwxrwx 0        0        7937-128-3 C:/WINDOWS/system32/drivers/usbuhci.sys
Tue Aug 03 2004 23:08:44    57600 m... r/rrwxrwxrwx 0        0        7935-128-3 C:/WINDOWS/system32/drivers/usbhub.sys
                           142976 m... r/rrwxrwxrwx 0        0        7936-128-3 C:/WINDOWS/system32/drivers/usbport.sys
Tue Aug 03 2004 23:08:48    26496 m... r/rrwxrwxrwx 0        0        7945-128-3 C:/WINDOWS/system32/drivers/USBSTOR.SYS
                            31616 m... r/rrwxrwxrwx 0        0        8687-128-3 C:/WINDOWS/system32/drivers/usbccgp.sys
Tue Aug 03 2004 23:14:38    52736 m... r/rrwxrwxrwx 0        0        8523-128-3 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0002102.sys
                            52736 m... r/rrwxrwxrwx 0        0        9131-128-3 C:/WINDOWS/system32/ReinstallBackups/0003/DriverFiles/i386/i8042prt.sys
                            52736 m... r/rrwxrwxrwx 0        0        9137-128-4 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0003122.sys
                            52736 m... r/rrwxrwxrwx 0        0        9138-128-3 C:/WINDOWS/system32/drivers/i8042prt.sys
Wed Aug 04 2004 00:56:46   464384 m... r/rrwxrwxrwx 0        0        9094-128-3 C:/WINDOWS/system32/spool/drivers/w32x86/3/PSCRIPT5.DLL
                           132608 m... r/rrwxrwxrwx 0        0        9095-128-3 C:/WINDOWS/system32/spool/drivers/w32x86/3/PS5UI.DLL
Wed Aug 04 2004 00:56:48    74240 m... r/rrwxrwxrwx 0        0        7934-128-3 C:/WINDOWS/system32/usbui.dll
Wed Aug 04 2004 01:01:08    40840 m... r/rrwxrwxrwx 0        0        4705-128-3 C:/WINDOWS/system32/drivers/termdd.sys
Wed Aug 04 2004 01:06:34     4821 m... r/rrwxrwxrwx 0        0        5318-128-3 C:/Program Files/Messenger/logowin.gif
                             7047 m... r/rrwxrwxrwx 0        0        5319-128-3 C:/Program Files/Messenger/lvback.gif
                            82944 m... r/rrwxrwxrwx 0        0        5320-128-3 C:/Program Files/Messenger/msgsc.dll
                           180224 m... r/rrwxrwxrwx 0        0        5321-128-3 C:/Program Files/Messenger/msgslang.dll
                          1667584 m... r/rrwxrwxrwx 0        0        5322-128-3 C:/Program Files/Messenger/msmsgs.exe
Wed Aug 04 2004 01:06:36     4454 m... r/rrwxrwxrwx 0        0        5323-128-3 C:/Program Files/Messenger/type.wav
                           115981 m... r/rrwxrwxrwx 0        0        5324-128-3 C:/Program Files/Messenger/xpmsgr.chm
Wed Aug 04 2004 06:58:30    60800 ...b r/rrwxrwxrwx 0        0        2344-128-3 C:/WINDOWS/system32/drivers/arp1394.sys
                            61824 ...b r/rrwxrwxrwx 0        0        2810-128-3 C:/WINDOWS/system32/drivers/nic1394.sys
Wed Aug 04 2004 06:58:34    23040 ...b r/rrwxrwxrwx 0        0        8522-128-3 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0002101.sys
                            23040 ...b r/rrwxrwxrwx 0        0        9136-128-3 C:/WINDOWS/system32/drivers/mouclass.sys
Wed Aug 04 2004 06:58:42     4352 ...b r/rrwxrwxrwx 0        0        2049-128-3 C:/WINDOWS/system32/drivers/swenum.sys
Wed Aug 04 2004 06:59:00  2056832 ...b r/rrwxrwxrwx 0        0        3477-128-3 C:/WINDOWS/system32/ntkrnlpa.exe

Open in new window

0
Comment
Question by:VMthinker
  • 11
  • 8
19 Comments
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 34165871
It is a bit unclear as to what your user input is. Is it a single begin date or is it a start and end date?
0
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 34165891
As for searching for the text, I believe string.StartsWith() will suit this situation:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.IO;

namespace Testing
{
    class Analysis
    {
        static void Main()
        {
            // Read the file lines into a string array.
            string[] lines = System.IO.File.ReadAllLines(@"C:\Test\ntfs.txt");

            System.Console.WriteLine("Analyzing ntfs.txt:");

            foreach (string line in lines)
            {
                Console.WriteLine("\t" + line);

                //  ***Trying to filter/grep out dates, file size, etc****
                if (line.StartsWith("Sun Nov 19 2000"))
                {
                    Console.WriteLine("Print entire line");
                }
            }

#if DEBUG
            // Keep the console window open in debug mode.
            Console.WriteLine("Press any key to exit.");
            System.Console.ReadKey();
#endif
        }
    }
}

Open in new window

0
 
LVL 2

Author Comment

by:VMthinker
ID: 34165985
Yea.... But what if some of the array has no start date on its lines array? This is due to multiple processes happening at the same time. Any ideas on that? Thanks.
0
ScreenConnect 6.0 Free Trial

Want empowering updates? You're in the right place! Discover new features in ScreenConnect 6.0, based on partner feedback, to keep you business operating smoothly and optimally (the way it should be). Explore all of the extras and enhancements for yourself!

 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 34166016
What is the answer to, "Is it a single begin date or is it a start and end date?"
0
 
LVL 2

Author Comment

by:VMthinker
ID: 34166057
Sorry about that. It is a start date to end date thing. The program is more to like a timeline thing. Thanks.
0
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 34166073
One more question, inclusive or exclusive dates?
0
 
LVL 2

Author Comment

by:VMthinker
ID: 34166163
I am a bit confuse about the inclusive or exclusive dates question but I need the dates to be used as the main form of greping/filtering the results like a timeline of the various system processes during the start to end date. So I guess its inclusive?
0
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 34166330
Inclusive vs exclusive means if you say start date is "Tue Aug 03 2004", then dates that fall on or after "Tue Aug 03 2004" will be printed. Exclusive means dates that only fall after "Tue Aug 03 2004" will be printed.

I am curious about one thing: when I think of "grep", I think of passing in any string (or pattern in some versions) and getting a list of lines containing that string. Here, I get the feeling you are creating a more specific search filter in that you want to print all lines in a range, where the range is set by your users. Maybe grep does this, but I'm not terribly experienced with it. Am I missing something?
0
 
LVL 2

Author Comment

by:VMthinker
ID: 34169332
I am also not sure about where should the parameters be thus I have added the "If" statement. The grep function that was originally thought of was actually due to the scripting knowledge I usually utilize in Unix scripts but I am unsure if that would work in C#. Do you have any website of sample codes that I can utilize that I can use C# date range within a log text files or could you tweak the codes above abit? Thanks.
0
 
LVL 75

Accepted Solution

by:
käµfm³d   👽 earned 500 total points
ID: 34172720
Here's what I'm thinking:  Take in a date range as two command line parameters (since you're thinking grep), validate the dates as valid, then return all text lines falling between and including those dates. Following is code describing this. I am passing dates in the form "mm/dd/yyyy" at the command line, but you should be able to use other formats as well. You can read up on the DateTime structure and its available methods to play around with how you receive your dates as parameters:  http://msdn.microsoft.com/en-us/library/system.datetime.aspx
class Program
{
    // Expected parameters:  1 - startDate; 2 - endDate; 3 - filename
    static void Main(string[] args)
    {
        if (args.Length >= 3)
        {
            DateTime startDate, endDate;

            if (DateTime.TryParse(args[0], out startDate) && DateTime.TryParse(args[1], out endDate))
            {
                if (System.IO.File.Exists(args[2]))
                {
                    string[] lines = System.IO.File.ReadAllLines(args[2]);
                    DateTime temp;
                    System.Globalization.CultureInfo ci = new System.Globalization.CultureInfo("en-US");
                    bool foundStartDate = false;

                    for (int i = 0; i < lines.Length; i++)
                    {
                        // DateTime.TryParseExact() allows you to pass a format string of the expected date.
                        //  It returns true if the value is a date or false otherwise
                        //
                        // "ddd MMM dd yyyy" this should match dates like "Tue Aug 03 2004"
                        if (DateTime.TryParseExact(lines[i].Substring(0, 15), "ddd MMM dd yyyy", ci, System.Globalization.DateTimeStyles.None, out temp))
                        {
                            if (foundStartDate)
                            {
                                if (temp >= endDate)
                                {
                                    break;
                                }
                                else
                                {
                                    Console.WriteLine(lines[i]);
                                }
                            }
                            else if (temp >= startDate)
                            {
                                foundStartDate = true;
                                Console.WriteLine(lines[i]);
                            }
                        }
                        else  // startDate has been found, but the date part may be blank, so we print this line
                              //  since it should be between start and end
                        {
                            if (foundStartDate)
                            {
                                Console.WriteLine(lines[i]);
                            }
                        }
                    }
                }
                else
                {
                    // Display invalid file and usage prompt
                }
            }
            else
            {
                // Display invalid date and usage prompt
            }
        }
        else
        {
            // Display usage prompt
        }
    }
}

Open in new window

0
 
LVL 2

Author Comment

by:VMthinker
ID: 34174501
Thanks for your reply kauf but do you think you can help me to tweak the codes you've shown me to time format instead?

I apologize for the extra work but I've managed to find a way to set a date limit to the original perl script needed to generate the logs. Now all thats left is to allow the user to select the time example "22:58:34" to "23:01:16" on the same day?

I really appriciate the help provided and hope to hear your reply.
0
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 34174678
As far as passing the time on the command line, you'll need to make sure your users know to quote it ( "02/04/2010 08:03:00" for example). This is a limitation of Windows (and others), not of C#. Other than that, the only modification you should need is at line 25. Change the format string from

    "ddd MMM dd yyyy"

to

    "ddd MMM dd yyyy hh:mm:ss"

and you should be set.
0
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 34174692
Note:

The "MMM" for month abbreviation and the "mm" for minutes are case-sensitive and should be entered as displayed.
0
 
LVL 2

Author Comment

by:VMthinker
ID: 34177997
Is it possible to only have the hh:mm:ss only as I think the script would be able to automatically filter out the date or do you recommend putting in the date as well? Thanks.
0
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 34179918
Have it where? When reading lines from the file or as input from the command line?
0
 
LVL 2

Author Comment

by:VMthinker
ID: 34180127
Thanks for your reply regarding your above question I guess that it would have to be both?

Just to ask is the codes provided suppose to run as an independent .exe program on the CMD? I tried to run it with these command on the CMD "Testing.exe Fri Nov 16 2000 Sat Aug 2003 C:\Test.ntfs"  with random dates not exact dates but it does not show any results except for a blank line. Is the error due to the spaces where the dates were placed in?

Or am I suppose yo use the class to be implemented inside my main program? If so where should I put the date parameters?

Once again thanks for your help.

0
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 34188963
This is the expected command line input:

    appname.exe "Fri Nov 16 2000" "Sat Aug 2003" "C:\Test.ntfs"

Note: You only need to quote command line parameters when they contain spaces. In the above example, "C:\Test.ntfs" does not have to be quoted since it has no spaces. Example:

    appname.exe "Fri Nov 16 2000" "Sat Aug 2003" C:\Test.ntfs
0
 
LVL 2

Author Closing Comment

by:VMthinker
ID: 34189061
Thanks kaufmed! Your codes and explination really helped!
0
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 34189154
Sure. Glad to help  :)
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
C# HTTP GET method sample code 3 57
Error in script 11 56
Reset asp.net login password 4 42
Video Player 11 23
Entity Framework is a powerful tool to help you interact with the DataBase but still doesn't help much when we have a Stored Procedure that returns more than one resultset. The solution takes some of out-of-the-box thinking; read on!
Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
THe viewer will learn how to use NetBeans IDE 8.0 for Windows to perform CRUD operations on a MySql database.
The viewer will learn how to use and create new code templates in NetBeans IDE 8.0 for Windows.

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question