Speed Up Execution of Perl LWP Module
Posted on 2004-04-19
I used to have a site on a shared server running Sun Solaris 5.8 and Perl 5.6.1. We have now moved to a dedicated Linux server running Red Hat Enterprise Linux V3, Cpanel/Web Host Manager (latest), and Perl 5.8.1. I have a perl script that collects images from a site on the web and downloads them to our server. Typcially processing between 100 to 1000 at a time. I've noticed a slight performance decrease since moving to the new RHEL server. When processing 30 urls it takes about 10 seconds longer then before. Although this difference is minor, the lag really shows up when processing thousands of images at a time. What are potential causes of the noticeable descrease in my Perl scripts performance when utilizing the LWP module?
What steps can I take to improve this.