VMthinker
asked on
C# Programming How to grep columns/lines from Text File?
Hi there! I have a C# console program which main functions should let a user grep lines / columns from a log text file.
An Example within the text file the user wishes to grep a group of all the related lines starting from a particular date etc. "Tue Aug 03 2004 22:58:34" to "Wed Aug 04 2004 00:56:48". Therefore after processing, the program would then output all the data found within the log text files between the 2 dates.
Could someone please advise on some codes that I could use to grep or create a filter to retrieve the neccessary text/data from the file? Thanks!
C# Program Files:
Log Text File Example:
An Example within the text file the user wishes to grep a group of all the related lines starting from a particular date etc. "Tue Aug 03 2004 22:58:34" to "Wed Aug 04 2004 00:56:48". Therefore after processing, the program would then output all the data found within the log text files between the 2 dates.
Could someone please advise on some codes that I could use to grep or create a filter to retrieve the neccessary text/data from the file? Thanks!
C# Program Files:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.IO;
namespace Testing
{
class Analysis
{
static void Main()
{
// Read the file lines into a string array.
string[] lines = System.IO.File.ReadAllLines(@"C:\Test\ntfs.txt");
System.Console.WriteLine("Analyzing ntfs.txt:");
foreach (string line in lines)
{
Console.WriteLine("\t" + line);
// ***Trying to filter/grep out dates, file size, etc****
if (lines = "Sun Nov 19 2000")
{
Console.WriteLine("Print entire line");
}
}
// Keep the console window open in debug mode.
Console.WriteLine("Press any key to exit.");
System.Console.ReadKey();
}
}
}Log Text File Example:
Fri May 14 2004 09:56:58 1581056 m... r/rrwxrwxrwx 0 0 8168-128-3 C:/Program Files/AccessData/AccessData Forensic Toolkit/Program/IsoBuster.dll
Tue May 25 2004 14:34:12 5233391 m... r/rrwxrwxrwx 0 0 8355-128-3 C:/Program Files/AccessData/AccessData Forensic Toolkit/FTKManual.pdf
Mon Jun 07 2004 08:00:00 0 .a.. r/rrwxrwxrwx 0 0 8626-128-1 C:/RECYCLER/S-1-5-21-606747145-1004336348-839522115-1003/Dc1.LOG
Tue Jun 08 2004 03:50:55 0 ...b r/rrwxrwxrwx 0 0 8626-128-1 C:/RECYCLER/S-1-5-21-606747145-1004336348-839522115-1003/Dc1.LOG
Tue Jun 08 2004 03:50:56 0 m... r/rrwxrwxrwx 0 0 8626-128-1 C:/RECYCLER/S-1-5-21-606747145-1004336348-839522115-1003/Dc1.LOG
Fri Jul 16 2004 15:49:34 352256 m... r/rrwxrwxrwx 0 0 8358-128-3 C:/Program Files/AccessData/Common Files/AccessData LanguageSelector/LanguageSelector.exe
Sat Jul 17 2004 11:41:52 792644 m... r/rrwxrwxrwx 0 0 9098-128-3 C:/WINDOWS/system32/spool/drivers/w32x86/3/PSCRIPT.NTF
Wed Jul 21 2004 16:58:48 499712 m... r/rrwxrwxrwx 0 0 8360-128-3 C:/Program Files/AccessData/Common Files/AccessData LicenseManager/LicenseManager.exe
Tue Aug 03 2004 22:58:34 23040 m... r/rrwxrwxrwx 0 0 8522-128-3 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0002101.sys
23040 m... r/rrwxrwxrwx 0 0 9132-128-3 C:/WINDOWS/system32/ReinstallBackups/0003/DriverFiles/i386/mouclass.sys
23040 m... r/rrwxrwxrwx 0 0 9135-128-4 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0003123.sys
23040 m... r/rrwxrwxrwx 0 0 9136-128-3 C:/WINDOWS/system32/drivers/mouclass.sys
Tue Aug 03 2004 23:01:16 196864 m... r/rrwxrwxrwx 0 0 4706-128-3 C:/WINDOWS/system32/drivers/rdpdr.sys
Tue Aug 03 2004 23:08:18 24960 m... r/rrwxrwxrwx 0 0 8690-128-3 C:/WINDOWS/system32/drivers/hidparse.sys
Tue Aug 03 2004 23:08:20 36224 m... r/rrwxrwxrwx 0 0 8689-128-3 C:/WINDOWS/system32/drivers/hidclass.sys
Tue Aug 03 2004 23:08:38 20480 m... r/rrwxrwxrwx 0 0 7937-128-3 C:/WINDOWS/system32/drivers/usbuhci.sys
Tue Aug 03 2004 23:08:44 57600 m... r/rrwxrwxrwx 0 0 7935-128-3 C:/WINDOWS/system32/drivers/usbhub.sys
142976 m... r/rrwxrwxrwx 0 0 7936-128-3 C:/WINDOWS/system32/drivers/usbport.sys
Tue Aug 03 2004 23:08:48 26496 m... r/rrwxrwxrwx 0 0 7945-128-3 C:/WINDOWS/system32/drivers/USBSTOR.SYS
31616 m... r/rrwxrwxrwx 0 0 8687-128-3 C:/WINDOWS/system32/drivers/usbccgp.sys
Tue Aug 03 2004 23:14:38 52736 m... r/rrwxrwxrwx 0 0 8523-128-3 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0002102.sys
52736 m... r/rrwxrwxrwx 0 0 9131-128-3 C:/WINDOWS/system32/ReinstallBackups/0003/DriverFiles/i386/i8042prt.sys
52736 m... r/rrwxrwxrwx 0 0 9137-128-4 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0003122.sys
52736 m... r/rrwxrwxrwx 0 0 9138-128-3 C:/WINDOWS/system32/drivers/i8042prt.sys
Wed Aug 04 2004 00:56:46 464384 m... r/rrwxrwxrwx 0 0 9094-128-3 C:/WINDOWS/system32/spool/drivers/w32x86/3/PSCRIPT5.DLL
132608 m... r/rrwxrwxrwx 0 0 9095-128-3 C:/WINDOWS/system32/spool/drivers/w32x86/3/PS5UI.DLL
Wed Aug 04 2004 00:56:48 74240 m... r/rrwxrwxrwx 0 0 7934-128-3 C:/WINDOWS/system32/usbui.dll
Wed Aug 04 2004 01:01:08 40840 m... r/rrwxrwxrwx 0 0 4705-128-3 C:/WINDOWS/system32/drivers/termdd.sys
Wed Aug 04 2004 01:06:34 4821 m... r/rrwxrwxrwx 0 0 5318-128-3 C:/Program Files/Messenger/logowin.gif
7047 m... r/rrwxrwxrwx 0 0 5319-128-3 C:/Program Files/Messenger/lvback.gif
82944 m... r/rrwxrwxrwx 0 0 5320-128-3 C:/Program Files/Messenger/msgsc.dll
180224 m... r/rrwxrwxrwx 0 0 5321-128-3 C:/Program Files/Messenger/msgslang.dll
1667584 m... r/rrwxrwxrwx 0 0 5322-128-3 C:/Program Files/Messenger/msmsgs.exe
Wed Aug 04 2004 01:06:36 4454 m... r/rrwxrwxrwx 0 0 5323-128-3 C:/Program Files/Messenger/type.wav
115981 m... r/rrwxrwxrwx 0 0 5324-128-3 C:/Program Files/Messenger/xpmsgr.chm
Wed Aug 04 2004 06:58:30 60800 ...b r/rrwxrwxrwx 0 0 2344-128-3 C:/WINDOWS/system32/drivers/arp1394.sys
61824 ...b r/rrwxrwxrwx 0 0 2810-128-3 C:/WINDOWS/system32/drivers/nic1394.sys
Wed Aug 04 2004 06:58:34 23040 ...b r/rrwxrwxrwx 0 0 8522-128-3 C:/System Volume Information/_restore{88D7369F-4F7E-44D4-8CD1-F7FF1F6AC067}/RP4/A0002101.sys
23040 ...b r/rrwxrwxrwx 0 0 9136-128-3 C:/WINDOWS/system32/drivers/mouclass.sys
Wed Aug 04 2004 06:58:42 4352 ...b r/rrwxrwxrwx 0 0 2049-128-3 C:/WINDOWS/system32/drivers/swenum.sys
Wed Aug 04 2004 06:59:00 2056832 ...b r/rrwxrwxrwx 0 0 3477-128-3 C:/WINDOWS/system32/ntkrnlpa.exe
kaufmed
It is a bit unclear as to what your user input is. Is it a single begin date or is it a start and end date?
As for searching for the text, I believe string.StartsWith() will suit this situation:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.IO;
namespace Testing
{
class Analysis
{
static void Main()
{
// Read the file lines into a string array.
string[] lines = System.IO.File.ReadAllLines(@"C:\Test\ntfs.txt");
System.Console.WriteLine("Analyzing ntfs.txt:");
foreach (string line in lines)
{
Console.WriteLine("\t" + line);
// ***Trying to filter/grep out dates, file size, etc****
if (line.StartsWith("Sun Nov 19 2000"))
{
Console.WriteLine("Print entire line");
}
}
#if DEBUG
// Keep the console window open in debug mode.
Console.WriteLine("Press any key to exit.");
System.Console.ReadKey();
#endif
}
}
}
ASKER
Yea.... But what if some of the array has no start date on its lines array? This is due to multiple processes happening at the same time. Any ideas on that? Thanks.
What is the answer to, "Is it a single begin date or is it a start and end date?"
ASKER
Sorry about that. It is a start date to end date thing. The program is more to like a timeline thing. Thanks.
One more question, inclusive or exclusive dates?
ASKER
I am a bit confuse about the inclusive or exclusive dates question but I need the dates to be used as the main form of greping/filtering the results like a timeline of the various system processes during the start to end date. So I guess its inclusive?
Inclusive vs exclusive means if you say start date is "Tue Aug 03 2004", then dates that fall on or after "Tue Aug 03 2004" will be printed. Exclusive means dates that only fall after "Tue Aug 03 2004" will be printed.
I am curious about one thing: when I think of "grep", I think of passing in any string (or pattern in some versions) and getting a list of lines containing that string. Here, I get the feeling you are creating a more specific search filter in that you want to print all lines in a range, where the range is set by your users. Maybe grep does this, but I'm not terribly experienced with it. Am I missing something?
I am curious about one thing: when I think of "grep", I think of passing in any string (or pattern in some versions) and getting a list of lines containing that string. Here, I get the feeling you are creating a more specific search filter in that you want to print all lines in a range, where the range is set by your users. Maybe grep does this, but I'm not terribly experienced with it. Am I missing something?
ASKER
I am also not sure about where should the parameters be thus I have added the "If" statement. The grep function that was originally thought of was actually due to the scripting knowledge I usually utilize in Unix scripts but I am unsure if that would work in C#. Do you have any website of sample codes that I can utilize that I can use C# date range within a log text files or could you tweak the codes above abit? Thanks.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks for your reply kauf but do you think you can help me to tweak the codes you've shown me to time format instead?
I apologize for the extra work but I've managed to find a way to set a date limit to the original perl script needed to generate the logs. Now all thats left is to allow the user to select the time example "22:58:34" to "23:01:16" on the same day?
I really appriciate the help provided and hope to hear your reply.
I apologize for the extra work but I've managed to find a way to set a date limit to the original perl script needed to generate the logs. Now all thats left is to allow the user to select the time example "22:58:34" to "23:01:16" on the same day?
I really appriciate the help provided and hope to hear your reply.
As far as passing the time on the command line, you'll need to make sure your users know to quote it ( "02/04/2010 08:03:00" for example). This is a limitation of Windows (and others), not of C#. Other than that, the only modification you should need is at line 25. Change the format string from
"ddd MMM dd yyyy"
to
"ddd MMM dd yyyy hh:mm:ss"
and you should be set.
"ddd MMM dd yyyy"
to
"ddd MMM dd yyyy hh:mm:ss"
and you should be set.
Note:
The "MMM" for month abbreviation and the "mm" for minutes are case-sensitive and should be entered as displayed.
The "MMM" for month abbreviation and the "mm" for minutes are case-sensitive and should be entered as displayed.
ASKER
Is it possible to only have the hh:mm:ss only as I think the script would be able to automatically filter out the date or do you recommend putting in the date as well? Thanks.
Have it where? When reading lines from the file or as input from the command line?
ASKER
Thanks for your reply regarding your above question I guess that it would have to be both?
Just to ask is the codes provided suppose to run as an independent .exe program on the CMD? I tried to run it with these command on the CMD "Testing.exe Fri Nov 16 2000 Sat Aug 2003 C:\Test.ntfs" with random dates not exact dates but it does not show any results except for a blank line. Is the error due to the spaces where the dates were placed in?
Or am I suppose yo use the class to be implemented inside my main program? If so where should I put the date parameters?
Once again thanks for your help.
Just to ask is the codes provided suppose to run as an independent .exe program on the CMD? I tried to run it with these command on the CMD "Testing.exe Fri Nov 16 2000 Sat Aug 2003 C:\Test.ntfs" with random dates not exact dates but it does not show any results except for a blank line. Is the error due to the spaces where the dates were placed in?
Or am I suppose yo use the class to be implemented inside my main program? If so where should I put the date parameters?
Once again thanks for your help.
This is the expected command line input:
appname.exe "Fri Nov 16 2000" "Sat Aug 2003" "C:\Test.ntfs"
Note: You only need to quote command line parameters when they contain spaces. In the above example, "C:\Test.ntfs" does not have to be quoted since it has no spaces. Example:
appname.exe "Fri Nov 16 2000" "Sat Aug 2003" C:\Test.ntfs
appname.exe "Fri Nov 16 2000" "Sat Aug 2003" "C:\Test.ntfs"
Note: You only need to quote command line parameters when they contain spaces. In the above example, "C:\Test.ntfs" does not have to be quoted since it has no spaces. Example:
appname.exe "Fri Nov 16 2000" "Sat Aug 2003" C:\Test.ntfs
ASKER
Thanks kaufmed! Your codes and explination really helped!
Sure. Glad to help :)