shragi
asked on
Perl efficient DB Call
I wrote the below script to update the file.
The script works but I had a small concern with the script I wrote, it makes multiple database calls.
Is there any way you think I can modify this and make just one DB call.
This is what the script currently does:
1) Go thru each line of the file and get the 3 fields that are required ($fields[6], $fields[8], $fields[12])
2) Run query and get the results using the 3 fields
3) Print output file line by line by appending the results at the end of each line.
Thanks,
The script works but I had a small concern with the script I wrote, it makes multiple database calls.
Is there any way you think I can modify this and make just one DB call.
This is what the script currently does:
1) Go thru each line of the file and get the 3 fields that are required ($fields[6], $fields[8], $fields[12])
2) Run query and get the results using the 3 fields
3) Print output file line by line by appending the results at the end of each line.
#!/usr/bin/perl
use 5.006;
use strict;
use warnings;
use Pod::Usage;
use Getopt::Long;
use Cwd;
use DateTime;
use DBD::mysql;
use DBI;
my $helpme = 0;
my $man = 0;
my $inputFileName = 'C:\\temp\\test.txt';
my $outputFileName = 'C:\\temp\\output.txt';
my $dsn = 'DBI:ODBC:Driver={SQL Server}';
my $host = 'testdb';
my $database = 'MYAPP';
my $user = 'admin';
my $auth = 'p@ssword';
GetOptions('help' => \$helpme, 'man' => \$man, 'infile=s' => \$inputFileName, 'outfile=s' => \$outputFileName) or pod2usage(2);
pod2usage(1) if $helpme;
pod2usage(-verbose => 2) if $man;
die "No input file name specified!" if !$inputFileName;
die "No output file name specified!" if !$outputFileName;
my $dbh = DBI->connect("$dsn;Server=$host;Database=$database", $user, $auth,{ RaiseError => 1, AutoCommit => 1}) || die "Database connection not made: $DBI::errstr";
open(INFILE, "<", $inputFileName) || die "Could not open input file!";
open(OUTFILE, ">", $outputFileName) || die "Could not open/create output file!";
while (<INFILE>) {
chomp;
my @fields = split /\s*\|\s*/;
my $sql = "SELECT e.id1, d.id2, s.id3
FROM Employee e
JOIN Department d
ON d.did = '$fields[12]' AND d.status ='A'
JOIN Student s
ON s.sid= '$fields[6]' AND s.status ='A'
Where e.eid= '$fields[8]' AND e.status ='A'";
my $sth = $dbh->prepare($sql);
$sth->execute();
while (my @row = $sth->fetchrow_array) {
print OUTFILE $_."|".$row[0]."|".$row[1]."|".$row[2]."\n";
}
}
close INFILE;
close OUTFILE;
Thanks,
ASKER
Sample file is something like the attached.
ASKER
ASKER
well if i want to append at the end of the file I can see that I create the sql first and execute it at the end, but i need to modify each line
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
I did not do any testing, but your join statements look a little fishy. Do they actually do what you need?
FishMonger is correct in most cases. However, there are databases where using placeholders is less efficient than executing a static sql each time (Sybase is (or at least was) one).
FishMonger is correct in most cases.What? You mean that I'm not perfect?
:-) lol ;-)
Can you post a reasonable sample of the input file?