perl dbi - limit cpu and memory usage

Posted on 2008-10-21
Last Modified: 2008-10-22
Hi, I have a Perl script that using DBI (DBD::CSV) to read a CSV file via a SELECT * to and convert it to a Excel file. Script is run on Windows 2003 server.

The script works correctly, but the Perl process and the Excel (via OLE) subprocess eat up ~100% of the CPU usage and a large amount of memory.  The CSV files are ~30MBs.

Is there a way to limit the CPU usage for the script? Or at a lower level (e.g. at the DBI level)?

Might be nice to limit how much memory the process uses too, but this it not as important as limiting the percent of CPU usage.

And I have thought about using the SQL LIMIT syntax and putting in wait steps which I believe would work, but I was hoping there was an easier way to handle this rather than complicating the program.
Question by:rmundkowsky
  • 2
LVL 29

Expert Comment

by:Michael W
ID: 22771934
When it comes to a Windows Server environment, the task manager will pretty much allow the application to eat as much memory and cpu time as there is available.

One option that might work would be to setup a Linux server w/ Samba (if you need physical access to the data file) and the Win32::OLE Perl module, thus allowing the Linux server to run the Perl script in a controlled manner.


Author Comment

ID: 22772072
Hmm, not sure if you could run Win32::OLE on Linux to control Excel.  Doubtful.

LVL 39

Accepted Solution

Adam314 earned 500 total points
ID: 22772521
In the windows task manager, you should set the priority to "BelowNormal" or "Low".  Then windows will take care of the rest.

Note that if there is nothing else running, this will still use 100% CPU - but as soon as there is something else to do (as long as it has a higher priority), windows will do that other thing.

You can even have the script change it's priority.  Here's an example:

Author Comment

ID: 22778686
Well, I guess this is a pretty good solution. I say this, because I was hoping for away to limit the CPU usage, but being able to set the priority to a lower level via a script should be good enough.


Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

On Microsoft Windows, if  when you click or type the name of a .pl file, you get an error "is not recognized as an internal or external command, operable program or batch file", then this means you do not have the .pl file extension associated with …
Restoring deleted objects in Active Directory has been a standard feature in Active Directory for many years, yet some admins may not know what is available.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This video shows how to remove a single email address from the Outlook 2010 Auto Suggestion memory. NOTE: For Outlook 2016 and 2013 perform the exact same steps. Open a new email: Click the New email button in Outlook. Start typing the address: …

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now