?
Solved

perl dbi - limit cpu and memory usage

Posted on 2008-10-21
4
Medium Priority
?
1,096 Views
Last Modified: 2008-10-22
Hi, I have a Perl script that using DBI (DBD::CSV) to read a CSV file via a SELECT * to and convert it to a Excel file. Script is run on Windows 2003 server.

The script works correctly, but the Perl process and the Excel (via OLE) subprocess eat up ~100% of the CPU usage and a large amount of memory.  The CSV files are ~30MBs.

Is there a way to limit the CPU usage for the script? Or at a lower level (e.g. at the DBI level)?

Might be nice to limit how much memory the process uses too, but this it not as important as limiting the percent of CPU usage.

And I have thought about using the SQL LIMIT syntax and putting in wait steps which I believe would work, but I was hoping there was an easier way to handle this rather than complicating the program.
0
Comment
Question by:rmundkowsky
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 29

Expert Comment

by:Michael Worsham
ID: 22771934
When it comes to a Windows Server environment, the task manager will pretty much allow the application to eat as much memory and cpu time as there is available.

One option that might work would be to setup a Linux server w/ Samba (if you need physical access to the data file) and the Win32::OLE Perl module, thus allowing the Linux server to run the Perl script in a controlled manner.

Reference:
http://search.cpan.org/~jdb/Win32-OLE-0.1709/lib/Win32/OLE.pm
0
 

Author Comment

by:rmundkowsky
ID: 22772072
Hmm, not sure if you could run Win32::OLE on Linux to control Excel.  Doubtful.

0
 
LVL 39

Accepted Solution

by:
Adam314 earned 1500 total points
ID: 22772521
In the windows task manager, you should set the priority to "BelowNormal" or "Low".  Then windows will take care of the rest.

Note that if there is nothing else running, this will still use 100% CPU - but as soon as there is something else to do (as long as it has a higher priority), windows will do that other thing.

You can even have the script change it's priority.  Here's an example:
http://techtasks.com/code/viewbookcode/567
0
 

Author Comment

by:rmundkowsky
ID: 22778686
Well, I guess this is a pretty good solution. I say this, because I was hoping for away to limit the CPU usage, but being able to set the priority to a lower level via a script should be good enough.

Thanks.
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Many time we need to work with multiple files all together. If its windows system then we can use some GUI based editor to accomplish our task. But what if you are on putty or have only CLI(Command Line Interface) as an option to  edit your files. I…
Restoring deleted objects in Active Directory has been a standard feature in Active Directory for many years, yet some admins may not know what is available.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans
Suggested Courses

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question