perl dbi - limit cpu and memory usage

Posted on 2008-10-21
Medium Priority
Last Modified: 2008-10-22
Hi, I have a Perl script that using DBI (DBD::CSV) to read a CSV file via a SELECT * to and convert it to a Excel file. Script is run on Windows 2003 server.

The script works correctly, but the Perl process and the Excel (via OLE) subprocess eat up ~100% of the CPU usage and a large amount of memory.  The CSV files are ~30MBs.

Is there a way to limit the CPU usage for the script? Or at a lower level (e.g. at the DBI level)?

Might be nice to limit how much memory the process uses too, but this it not as important as limiting the percent of CPU usage.

And I have thought about using the SQL LIMIT syntax and putting in wait steps which I believe would work, but I was hoping there was an easier way to handle this rather than complicating the program.
Question by:rmundkowsky
  • 2
LVL 29

Expert Comment

by:Michael Worsham
ID: 22771934
When it comes to a Windows Server environment, the task manager will pretty much allow the application to eat as much memory and cpu time as there is available.

One option that might work would be to setup a Linux server w/ Samba (if you need physical access to the data file) and the Win32::OLE Perl module, thus allowing the Linux server to run the Perl script in a controlled manner.


Author Comment

ID: 22772072
Hmm, not sure if you could run Win32::OLE on Linux to control Excel.  Doubtful.

LVL 39

Accepted Solution

Adam314 earned 1500 total points
ID: 22772521
In the windows task manager, you should set the priority to "BelowNormal" or "Low".  Then windows will take care of the rest.

Note that if there is nothing else running, this will still use 100% CPU - but as soon as there is something else to do (as long as it has a higher priority), windows will do that other thing.

You can even have the script change it's priority.  Here's an example:

Author Comment

ID: 22778686
Well, I guess this is a pretty good solution. I say this, because I was hoping for away to limit the CPU usage, but being able to set the priority to a lower level via a script should be good enough.


Featured Post

Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

ADCs have gained traction within the last decade, largely due to increased demand for legacy load balancing appliances to handle more advanced application delivery requirements and improve application performance.
Restoring deleted objects in Active Directory has been a standard feature in Active Directory for many years, yet some admins may not know what is available.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans
Suggested Courses

621 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question