?
Solved

Compare values from text in 2 files

Posted on 2009-05-01
4
Medium Priority
?
157 Views
Last Modified: 2012-05-06

I have a requirement to parse through a file to get a file name and a number. I would like to be able to run the script by entering scriptname.pl filename to process (ex: myscript.pl testfile.dat).  
The file contents of testfile.dat look like this;

1)
File Name|Record Count
test_profile_affiliation_123.dat|24
test_profile_practice_123.dat|4
test_profile_education_123.dat|3

So in this example I have 3 file names and 3 numeric value pairs. There may be only one or many similar lines in the file.

2)
Once I have this information I need to check the contents of a second file which is located in the same directory as testfile.dat. The second file(s) will have the same file name as the one(s) extracted from step 1 except that the files have a .log extension instead of .dat.
So for the first line in the example I need to find file test_profile_affiliation_123.log
Next I need to search the contents of test_profile_affiliation_123.log until I find the string
 '24 rows successfully loaded'  From this string I need to capture the numeric value (24) and compare it to the numeric value from line 1, step 1.

3)
Finally, for each file name processed I need to create a text file in the same directory as a record that the numeric values did or didn't match between step 1 and step 2. So if the first record is
test_profile_affiliation_123.dat|24
And we find the matching log file;
test_profile_affiliation_123.log
Look for the number in the string '24 Rows successfully loaded' and compare to the numeric value from step 1.
Finally, Create file
test_profile_affiliation_123_pass.txt  OR test_profile_affiliation_123_fail.txt if 24 did not exist in both files.

4) Process next file name from step 1 or end if no more.

0
Comment
Question by:Ed_CLP
  • 2
  • 2
4 Comments
 
LVL 39

Expert Comment

by:Adam314
ID: 24282098

#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
 
<>;  #get rid of header
while(<>) {
	chomp;
	my ($fn, $num) = split /\|/;
	$fn =~ s/\.dat$//;
	open(my $in, "<$fn.dat") or warn "Could not open $fn: $!, skipping\n",next;
	while(<$in>) {
		next unless /(\d+)\s+rows successfully loaded/;
		if($1 == $num) { CreateFile("${fn}_pass.txt"); }
		else { CreateFile("${fn}_fail.txt"); }
	}
}
 
sub CreateFile {
	open(my $out, ">$_[0]") or warn "Could not create file $_[0]: $!\n",return;
	close($out);
}

Open in new window

0
 

Author Comment

by:Ed_CLP
ID: 24282553
I tried it, but it didn't seem to do anything.  I have attached the files I used to test it (renamed myscript.pl and test_control.dat to *.txt so I could upload them).
 Put these files in a directory and run myscript.pl test_control.dat to reproduce what I am trying to do.

Thanks for the fast response!
Sample-Files.zip
0
 
LVL 39

Accepted Solution

by:
Adam314 earned 2000 total points
ID: 24282793
The text for "rows successfully loaded" needed to have rows capitalized, while the script looked for it lower case.  Here is an update, with a few minor updates...
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
 
<>;  #get rid of header
while(<>) {
	chomp;
	my ($fn, $num) = split /\|/;
	$fn =~ s/\.dat$//;
	open(my $in, "<$fn.log") or warn("Could not open $fn: $!, skipping\n"),next;
	
	my $found = 0;
	while(<$in>) {
		next unless /(\d+) rows successfully loaded/i;
		$found=1;
		if($1 == $num) { CreateFile("${fn}_pass.txt"); }
		else { CreateFile("${fn}_fail.txt"); }
		last;
	}
	warn "Did not find \"rows successfully loaded\" for $fn\n" unless $found;
}
 
sub CreateFile {
        open(my $out, ">$_[0]") or warn "Could not create file $_[0]: $!\n",return;
        close($out);
}

Open in new window

0
 

Author Closing Comment

by:Ed_CLP
ID: 31576995
Excellent job!
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I've just discovered very important differences between Windows an Unix formats in Perl,at least 5.xx.. MOST IMPORTANT: Use Unix file format while saving Your script. otherwise it will have ^M s or smth likely weird in the EOL, Then DO NOT use m…
In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans
Suggested Courses

850 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question