Solved

Perl Pattern searching help with ftp flows

Posted on 2014-04-01
3
361 Views
Last Modified: 2014-04-02
I apologize in advance for my ignorance in perl, I'm unfortunately a beginner and am attempting teach myself as I go along with some video tutorials and books and help from you guys.

Ok, so I wrote a script in bash to show the entire flow of a ftp connection by searching username or IP address. I had it read the data into an array, search for criteria, and then match that process id with others so I would get the entire flow.

The performance however was extremely slow and from suggestion of others on the experts exchange community I decided to give it a try in perl.  I am attempting to learn as much as I can but still have a long way to go. I'm attempting to search for criteria, take the process id of that line, and then read all the lines into an array that matches that process id so I'm basically getting the entire flow of the ftp connection.  

I'm assuming I would read each line in from the file, do a pattern match on it and if it matches to the IP address that I'm searching for I would then copy that line to an array.  I'm then thinking that after I read those lines into the array I'll go back and grab the process id from each of those lines, do another search on the file and put all the lines matching the process id into a new array, and then print the array out.  

Does this sound about right? Any suggestions would help. Thanks.

examples of data in log file:
Dec  1 23:59:03 sslmftp1 ftpd[4152]: USER xxxxxx  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: PASS password  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: FTP LOGIN FROM 172.19.x.xx [172.19.x.xx], xxxxxx  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: PWD  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: CWD /test/data/872507/  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: TYPE Image

Open in new window

0
Comment
Question by:dloszewski
3 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 39971767
I'm not sure what you are wanting to do with that log file.
Can you give an example of what the result of processing that data should look like?
0
 
LVL 26

Accepted Solution

by:
wilcoxon earned 500 total points
ID: 39972316
Personally, I'd use Tie::File.
use strict;
use warnings;
use Tie::File;
use Fcntl 'O_RDONLY';
my $log = shift or die "Usage: $0 logfile ip\n";
my $ip = shift or die "Usage: $0 logfile ip\n";
tie my @file, 'Tie::File', $log, mode => O_RDONLY or die "could not open $log: $!";
# get pids
my %tmp;
foreach my $pid (map { / ftpd\[(\d+)\]/; $1 } grep /\Q$ip\E/, @file) {
    $tmp{$pid}++;
}
my @tmp = sort { $a <=> $b } keys %tmp;
print "found multiple pids (@tmp) for $ip\n" if (@tmp > 1);
# get log rows based on pid
my $rx = join '|', @tmp;
my @rows = grep / ftpd\[(?:$rx)\]/, @file;
print @rows;
# do whatever else you want with the rows for the ip specified

Open in new window

0
 

Author Closing Comment

by:dloszewski
ID: 39972338
Awesome, thanks!
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Over the years I've spent many an hour playing on hardened, DMZ'd servers, with only a sub-set of the usual GNU toy's to keep me company; frequently I've needed to save and send log or data extracts from these server back to my PC, or to others, and…
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now