perl dbi - limit cpu and memory usage

Posted on 2008-10-21
Last Modified: 2008-10-22
Hi, I have a Perl script that using DBI (DBD::CSV) to read a CSV file via a SELECT * to and convert it to a Excel file. Script is run on Windows 2003 server.

The script works correctly, but the Perl process and the Excel (via OLE) subprocess eat up ~100% of the CPU usage and a large amount of memory.  The CSV files are ~30MBs.

Is there a way to limit the CPU usage for the script? Or at a lower level (e.g. at the DBI level)?

Might be nice to limit how much memory the process uses too, but this it not as important as limiting the percent of CPU usage.

And I have thought about using the SQL LIMIT syntax and putting in wait steps which I believe would work, but I was hoping there was an easier way to handle this rather than complicating the program.
Question by:rmundkowsky
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
LVL 29

Expert Comment

by:Michael Worsham
ID: 22771934
When it comes to a Windows Server environment, the task manager will pretty much allow the application to eat as much memory and cpu time as there is available.

One option that might work would be to setup a Linux server w/ Samba (if you need physical access to the data file) and the Win32::OLE Perl module, thus allowing the Linux server to run the Perl script in a controlled manner.


Author Comment

ID: 22772072
Hmm, not sure if you could run Win32::OLE on Linux to control Excel.  Doubtful.

LVL 39

Accepted Solution

Adam314 earned 500 total points
ID: 22772521
In the windows task manager, you should set the priority to "BelowNormal" or "Low".  Then windows will take care of the rest.

Note that if there is nothing else running, this will still use 100% CPU - but as soon as there is something else to do (as long as it has a higher priority), windows will do that other thing.

You can even have the script change it's priority.  Here's an example:

Author Comment

ID: 22778686
Well, I guess this is a pretty good solution. I say this, because I was hoping for away to limit the CPU usage, but being able to set the priority to a lower level via a script should be good enough.


Featured Post

Free NetCrunch network monitor licenses!

Only on Experts-Exchange: Sign-up for a free-trial and we'll send you your permanent license!

Here is what you get: 30 Nodes | Unlimited Sensors | No Time Restrictions | Absolutely FREE!

Act now. This offer ends July 14, 2017.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Scenerio: You have a server running Server 2003 and have applied a retail pack of Terminal Server Licenses.  You want to change servers or your server has crashed and you need to reapply the Terminal Server Licenses. When you enter the 16-digit lic…
This article provides a convenient collection of links to Microsoft provided Security Patches for operating systems that have reached their End of Life support cycle. Included operating systems covered by this article are Windows XP,  Windows Server…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question