Solved

speed up slow Perl code with SQL statements

Posted on 2007-03-30
1
175 Views
Last Modified: 2010-03-05
This is the guts of the code that I have written, however, when displayed in Firefox it takes a while to display. This seems to be down to the SQL hits i think, is there a way to optimize it as best as possible? I tried a JOIN which works, but is there any other ways to help speed it up?

I feel the speed is slowing at the @filescursor query.

I'm using use strict;

@groupscursor = $db->query("select distinct(filename),groups.receiverid,groups.filelimit,groups.theowner,groups.active,receivers.name,receivers.extension from hols.groups left join hols.receivers on groups.receiverid = receivers.id order by theowner asc, receiverid asc");

if ($groupscursor[1] ne "0E0") {
      while ($groupsreturned = $groupscursor[0]->fetchrow_hashref) {
            $receiverid            = $groupsreturned->{receiverid};
            $filename            = $groupsreturned->{filename};
            $theowner            = $groupsreturned->{theowner};
            $receivername      = $groupsreturned->{name};
            $extension            = $groupsreturned->{extension};      

            next if ($receivername eq $tmpreceiver and ($filename.".".$extension) eq $tmpfile);


            @filescursor = $db->query("select count(id) as results,id,filename,completed,max(created) as datecreated from picco.files where theowner = '$theowner' and filename = '$filename' group by filename");            

            if ($filescursor[1] ne "0E0") {
                  while ($filesreturned = $filescursor[0]->fetchrow_hashref) {
                                                
                        $fileid            = $filesreturned->{id};
                        $created      = $filesreturned->{datecreated};
                        $completed      = $filesreturned->{completed};

                        $total += $filesreturned->{results};

                        ($offers) = @{($db->query("SELECT count(id) as offers FROM picco.offers WHERE fileid = '$fileid'"))[0]->fetchrow_arrayref};

                        if ($theowner ne $tmpsite) {                  
                              push (@list, ["", qq|<b>| . $theowner . qq|</b>|, "&nbsp;", "&nbsp;", "&nbsp;", "&nbsp;", "&nbsp;", "&nbsp;", "&nbsp;"]);
                              $total += 1;
                        }
                        $tmpsite = $theowner;
                        
                        $colour = "black";

                        if ($completed eq "N") {
                              $colour = "red";
                        }                        

                        push (@list,      [
                                                      "&nbsp;",                        
                                                      "&nbsp;",
                                                      qq|<font color='$colour'>| . $receivername . qq|</font>|,
                                                      qq|<font color='$colour'>| . ($filename .".". $extension) . qq|</font>|,
                                                      qq|<font color='$colour'>| . $groupsreturned->{filelimit} . qq|</font>|,
                                                      qq|<font color='$colour'>| . $groupsreturned->{active} . qq|</font>|,
                                                      qq|<font color='$colour'>| . $created . qq|</font>|,
                                                      qq|<font color='$colour'>| . $completed . qq|</font>|,
                                                      qq|<font color='$colour'>| . $offers . qq|</font>|
                                                ]);
                        $tmpfile = ($filename.".".$extension);
                        $tmpreceiver = $receivername;
                  }
            }
            @filescursor = undef;
      }
}
@groupscursor = undef;
0
Comment
Question by:crmpicco
1 Comment
 
LVL 7

Accepted Solution

by:
mzalfres earned 125 total points
ID: 18881503
I believe your problem lies not in the PERL code. Try to compare 'clean' SQL query run with your script. Run the same query using your database command line interface and see whether it works significantly faster. I'd rather look at database and try eg. add indexes for fields which are used as a search / join keys. (in this case - 'filename' field.)

Regards,

Marek ZJ.
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

On Microsoft Windows, if  when you click or type the name of a .pl file, you get an error "is not recognized as an internal or external command, operable program or batch file", then this means you do not have the .pl file extension associated with …
I have been pestered over the years to produce and distribute regular data extracts, and often the request have explicitly requested the data be emailed as an Excel attachement; specifically Excel, as it appears: CSV files confuse (no Red or Green h…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

839 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question