Solved

Perl Pattern searching help with ftp flows

Posted on 2014-04-01
3
362 Views
Last Modified: 2014-04-02
I apologize in advance for my ignorance in perl, I'm unfortunately a beginner and am attempting teach myself as I go along with some video tutorials and books and help from you guys.

Ok, so I wrote a script in bash to show the entire flow of a ftp connection by searching username or IP address. I had it read the data into an array, search for criteria, and then match that process id with others so I would get the entire flow.

The performance however was extremely slow and from suggestion of others on the experts exchange community I decided to give it a try in perl.  I am attempting to learn as much as I can but still have a long way to go. I'm attempting to search for criteria, take the process id of that line, and then read all the lines into an array that matches that process id so I'm basically getting the entire flow of the ftp connection.  

I'm assuming I would read each line in from the file, do a pattern match on it and if it matches to the IP address that I'm searching for I would then copy that line to an array.  I'm then thinking that after I read those lines into the array I'll go back and grab the process id from each of those lines, do another search on the file and put all the lines matching the process id into a new array, and then print the array out.  

Does this sound about right? Any suggestions would help. Thanks.

examples of data in log file:
Dec  1 23:59:03 sslmftp1 ftpd[4152]: USER xxxxxx  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: PASS password  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: FTP LOGIN FROM 172.19.x.xx [172.19.x.xx], xxxxxx  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: PWD  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: CWD /test/data/872507/  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: TYPE Image

Open in new window

0
Comment
Question by:dloszewski
3 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 39971767
I'm not sure what you are wanting to do with that log file.
Can you give an example of what the result of processing that data should look like?
0
 
LVL 26

Accepted Solution

by:
wilcoxon earned 500 total points
ID: 39972316
Personally, I'd use Tie::File.
use strict;
use warnings;
use Tie::File;
use Fcntl 'O_RDONLY';
my $log = shift or die "Usage: $0 logfile ip\n";
my $ip = shift or die "Usage: $0 logfile ip\n";
tie my @file, 'Tie::File', $log, mode => O_RDONLY or die "could not open $log: $!";
# get pids
my %tmp;
foreach my $pid (map { / ftpd\[(\d+)\]/; $1 } grep /\Q$ip\E/, @file) {
    $tmp{$pid}++;
}
my @tmp = sort { $a <=> $b } keys %tmp;
print "found multiple pids (@tmp) for $ip\n" if (@tmp > 1);
# get log rows based on pid
my $rx = join '|', @tmp;
my @rows = grep / ftpd\[(?:$rx)\]/, @file;
print @rows;
# do whatever else you want with the rows for the ip specified

Open in new window

0
 

Author Closing Comment

by:dloszewski
ID: 39972338
Awesome, thanks!
0

Featured Post

Courses: Start Training Online With Pros, Today

Brush up on the basics or master the advanced techniques required to earn essential industry certifications, with Courses. Enroll in a course and start learning today. Training topics range from Android App Dev to the Xen Virtualization Platform.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This Windows batch file is useful for organizing image files from a digital camera or other source, but can have many other uses.  It simply renames the file(s) to match their create date.  For example, if you took a picture today at 1:40pm and the …
In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

785 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question