Solved

Perl Pattern searching help with ftp flows

Posted on 2014-04-01
3
360 Views
Last Modified: 2014-04-02
I apologize in advance for my ignorance in perl, I'm unfortunately a beginner and am attempting teach myself as I go along with some video tutorials and books and help from you guys.

Ok, so I wrote a script in bash to show the entire flow of a ftp connection by searching username or IP address. I had it read the data into an array, search for criteria, and then match that process id with others so I would get the entire flow.

The performance however was extremely slow and from suggestion of others on the experts exchange community I decided to give it a try in perl.  I am attempting to learn as much as I can but still have a long way to go. I'm attempting to search for criteria, take the process id of that line, and then read all the lines into an array that matches that process id so I'm basically getting the entire flow of the ftp connection.  

I'm assuming I would read each line in from the file, do a pattern match on it and if it matches to the IP address that I'm searching for I would then copy that line to an array.  I'm then thinking that after I read those lines into the array I'll go back and grab the process id from each of those lines, do another search on the file and put all the lines matching the process id into a new array, and then print the array out.  

Does this sound about right? Any suggestions would help. Thanks.

examples of data in log file:
Dec  1 23:59:03 sslmftp1 ftpd[4152]: USER xxxxxx  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: PASS password  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: FTP LOGIN FROM 172.19.x.xx [172.19.x.xx], xxxxxx  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: PWD  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: CWD /test/data/872507/  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: TYPE Image

Open in new window

0
Comment
Question by:dloszewski
3 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 39971767
I'm not sure what you are wanting to do with that log file.
Can you give an example of what the result of processing that data should look like?
0
 
LVL 26

Accepted Solution

by:
wilcoxon earned 500 total points
ID: 39972316
Personally, I'd use Tie::File.
use strict;
use warnings;
use Tie::File;
use Fcntl 'O_RDONLY';
my $log = shift or die "Usage: $0 logfile ip\n";
my $ip = shift or die "Usage: $0 logfile ip\n";
tie my @file, 'Tie::File', $log, mode => O_RDONLY or die "could not open $log: $!";
# get pids
my %tmp;
foreach my $pid (map { / ftpd\[(\d+)\]/; $1 } grep /\Q$ip\E/, @file) {
    $tmp{$pid}++;
}
my @tmp = sort { $a <=> $b } keys %tmp;
print "found multiple pids (@tmp) for $ip\n" if (@tmp > 1);
# get log rows based on pid
my $rx = join '|', @tmp;
my @rows = grep / ftpd\[(?:$rx)\]/, @file;
print @rows;
# do whatever else you want with the rows for the ip specified

Open in new window

0
 

Author Closing Comment

by:dloszewski
ID: 39972338
Awesome, thanks!
0

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

Suggested Solutions

A year or so back I was asked to have a play with MongoDB; within half an hour I had downloaded (http://www.mongodb.org/downloads),  installed and started the daemon, and had a console window open. After an hour or two of playing at the command …
How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now