speed up slow Perl code with SQL statements

Posted on 2007-03-30
Last Modified: 2010-03-05
This is the guts of the code that I have written, however, when displayed in Firefox it takes a while to display. This seems to be down to the SQL hits i think, is there a way to optimize it as best as possible? I tried a JOIN which works, but is there any other ways to help speed it up?

I feel the speed is slowing at the @filescursor query.

I'm using use strict;

@groupscursor = $db->query("select distinct(filename),groups.receiverid,groups.filelimit,groups.theowner,,,receivers.extension from hols.groups left join hols.receivers on groups.receiverid = order by theowner asc, receiverid asc");

if ($groupscursor[1] ne "0E0") {
      while ($groupsreturned = $groupscursor[0]->fetchrow_hashref) {
            $receiverid            = $groupsreturned->{receiverid};
            $filename            = $groupsreturned->{filename};
            $theowner            = $groupsreturned->{theowner};
            $receivername      = $groupsreturned->{name};
            $extension            = $groupsreturned->{extension};      

            next if ($receivername eq $tmpreceiver and ($filename.".".$extension) eq $tmpfile);

            @filescursor = $db->query("select count(id) as results,id,filename,completed,max(created) as datecreated from picco.files where theowner = '$theowner' and filename = '$filename' group by filename");            

            if ($filescursor[1] ne "0E0") {
                  while ($filesreturned = $filescursor[0]->fetchrow_hashref) {
                        $fileid            = $filesreturned->{id};
                        $created      = $filesreturned->{datecreated};
                        $completed      = $filesreturned->{completed};

                        $total += $filesreturned->{results};

                        ($offers) = @{($db->query("SELECT count(id) as offers FROM picco.offers WHERE fileid = '$fileid'"))[0]->fetchrow_arrayref};

                        if ($theowner ne $tmpsite) {                  
                              push (@list, ["", qq|<b>| . $theowner . qq|</b>|, "&nbsp;", "&nbsp;", "&nbsp;", "&nbsp;", "&nbsp;", "&nbsp;", "&nbsp;"]);
                              $total += 1;
                        $tmpsite = $theowner;
                        $colour = "black";

                        if ($completed eq "N") {
                              $colour = "red";

                        push (@list,      [
                                                      qq|<font color='$colour'>| . $receivername . qq|</font>|,
                                                      qq|<font color='$colour'>| . ($filename .".". $extension) . qq|</font>|,
                                                      qq|<font color='$colour'>| . $groupsreturned->{filelimit} . qq|</font>|,
                                                      qq|<font color='$colour'>| . $groupsreturned->{active} . qq|</font>|,
                                                      qq|<font color='$colour'>| . $created . qq|</font>|,
                                                      qq|<font color='$colour'>| . $completed . qq|</font>|,
                                                      qq|<font color='$colour'>| . $offers . qq|</font>|
                        $tmpfile = ($filename.".".$extension);
                        $tmpreceiver = $receivername;
            @filescursor = undef;
@groupscursor = undef;
Question by:crmpicco
1 Comment

Accepted Solution

mzalfres earned 125 total points
ID: 18881503
I believe your problem lies not in the PERL code. Try to compare 'clean' SQL query run with your script. Run the same query using your database command line interface and see whether it works significantly faster. I'd rather look at database and try eg. add indexes for fields which are used as a search / join keys. (in this case - 'filename' field.)


Marek ZJ.

Featured Post

Announcing the Most Valuable Experts of 2016

MVEs are more concerned with the satisfaction of those they help than with the considerable points they can earn. They are the types of people you feel privileged to call colleagues. Join us in honoring this amazing group of Experts.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

On Microsoft Windows, if  when you click or type the name of a .pl file, you get an error "is not recognized as an internal or external command, operable program or batch file", then this means you do not have the .pl file extension associated with …
I have been pestered over the years to produce and distribute regular data extracts, and often the request have explicitly requested the data be emailed as an Excel attachement; specifically Excel, as it appears: CSV files confuse (no Red or Green h…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
In a recent question ( here at Experts Exchange, a member asked how to add page numbers to a PDF file using Adobe Acrobat XI Pro. This short video Micro Tutorial sh…

813 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

8 Experts available now in Live!

Get 1:1 Help Now