perl dbi - limit cpu and memory usage

Posted on 2008-10-21
Last Modified: 2008-10-22
Hi, I have a Perl script that using DBI (DBD::CSV) to read a CSV file via a SELECT * to and convert it to a Excel file. Script is run on Windows 2003 server.

The script works correctly, but the Perl process and the Excel (via OLE) subprocess eat up ~100% of the CPU usage and a large amount of memory.  The CSV files are ~30MBs.

Is there a way to limit the CPU usage for the script? Or at a lower level (e.g. at the DBI level)?

Might be nice to limit how much memory the process uses too, but this it not as important as limiting the percent of CPU usage.

And I have thought about using the SQL LIMIT syntax and putting in wait steps which I believe would work, but I was hoping there was an easier way to handle this rather than complicating the program.
Question by:rmundkowsky
  • 2
LVL 29

Expert Comment

by:Michael W
ID: 22771934
When it comes to a Windows Server environment, the task manager will pretty much allow the application to eat as much memory and cpu time as there is available.

One option that might work would be to setup a Linux server w/ Samba (if you need physical access to the data file) and the Win32::OLE Perl module, thus allowing the Linux server to run the Perl script in a controlled manner.


Author Comment

ID: 22772072
Hmm, not sure if you could run Win32::OLE on Linux to control Excel.  Doubtful.

LVL 39

Accepted Solution

Adam314 earned 500 total points
ID: 22772521
In the windows task manager, you should set the priority to "BelowNormal" or "Low".  Then windows will take care of the rest.

Note that if there is nothing else running, this will still use 100% CPU - but as soon as there is something else to do (as long as it has a higher priority), windows will do that other thing.

You can even have the script change it's priority.  Here's an example:

Author Comment

ID: 22778686
Well, I guess this is a pretty good solution. I say this, because I was hoping for away to limit the CPU usage, but being able to set the priority to a lower level via a script should be good enough.


Featured Post

NAS Cloud Backup Strategies

This article explains backup scenarios when using network storage. We review the so-called “3-2-1 strategy” and summarize the methods you can use to send NAS data to the cloud

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Numerous times I have been asked this questions that what is it that makes my machine log on so slow, there have been cases where computers took 23 minute exactly after taking password and getting to the desktop. Interesting thing was the fact th…
Setting up a Microsoft WSUS update system is free relatively speaking if you have hard disk space and processor capacity.   However, WSUS can be a blessing and a curse. For example, there is nothing worse than approving updates and they just have…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

943 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now