?
Solved

Memory Error

Posted on 1998-12-05
17
Medium Priority
?
188 Views
Last Modified: 2010-03-05
My perl program builds an HTML Table from data it pulls from oracle.  Is there any way I can either avoid the error or catch it and end the table before the process abruptly blows up?

Thanks
0
Comment
Question by:annakellers_pg
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 8
  • 5
  • 2
  • +1
17 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 1206748
If you can post the code, perhaps we can suggest a way to use your memory more efficiently
0
 

Author Comment

by:annakellers_pg
ID: 1206749
The code works fine for about 20,000 rows.  What I need is a method that checks memory.  Basically I cycle (fetch) though a cursor of data pulled from an oracle database building an HTML table.  The problem is that the PC runs out of memory when my customer asks for too much (like someone asked for data yesterday that pull 1,200,000 .. that is over a million rows).  Is there  a method out there to get the current level of memory ... if it is low I can issue a print "</TALBE>" and tell the user memory is maxed.  Do you know of a method/function out there I can use for memory safety nets.

Thanks

0
 

Author Comment

by:annakellers_pg
ID: 1206750
Please help
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 84

Expert Comment

by:ozo
ID: 1206751
It sounds like an obvious solution would be to stop at 20,000 rows
But it's still not clear why you need all 20000 rows in memory at once.
If your perl was compiled with -DPERL_EMERGENCY_SBRK, you might try setting $^M
0
 

Author Comment

by:annakellers_pg
ID: 1206752
I need a function/subroutine that checks memory.

POINTS UPED
Anna
0
 

Author Comment

by:annakellers_pg
ID: 1206753
I've upped my points as hight as possible.

I need to check pc resources.  Is there a system method/function to check memory or a subroutine that someone has used to check memory.

Please help.  I've given all the points I can spare.

Pleaseeeeeeeeeeeeeeee
0
 

Expert Comment

by:orchid2
ID: 1206754
If I understand correctly, people use your program (presumably cgi) to query a database.  Then the results are supposed to be returned to them in an HTML table.  Somtimes the results are huge and can deplete the client machine of memory.

If this is correct, it seems to me that you should make your program recieve the query from the database, but only return a reasonable number of results to the user (say, 100 or possibly 1000), with the option to go to the "next page" to see the next 100 or 1000 results.  Or, maybe just return a summary of the query results.

If the users really need ALL the results all at once (for example if they want to drop the results into a spreadsheet or another  program), you could have the query returned as a file on the server that the user could download without opening it in their browser where they'll have memory problems.

Does this make sense?

-Orchid2
0
 

Author Comment

by:annakellers_pg
ID: 1206755
The following is a good very-high level solution.  Can you give me some code example.
Here is your reply:

[Only return a reasonable number of results to the user (say, 100 or possibly 1000), with
the option to go to the "next page" to see the next 100 or 1000 results.  Or, maybe just return a summary of the query results. ]

Here is our code loop as of today:

   print "<TABLE BORDER=2>\n";
   $sql = "SELECT * FROM MYtable"
   $cursor = $db->prepare($sql) or die "Unable to Execute SQL: $DBI::errstr";
   $cursor->execute or die "Unable to Execute SQL: $DBI::errstr" ;
   while (@row = $cursor->fetchrow_array)
   {
      print "<TR>";
      foreach $Fields (@row)
      {
         print "<TD ALIGN=RIGHT>$Fields</TD>";
      }
      print "</TR>";
      $countit++;
   }
   print "</TABLE>";
}

Points Increased
Thanks

0
 

Expert Comment

by:orchid2
ID: 1206756
Hmmm.  Well, I'm new to experts-exchange.  But quite honestly and in all fairness, it seems to me that my answer, though high level, was appropriate to the question (which was rather general) and accurate.  I think you should give me the points for the answer you've received.

If you want help writing the actual code, I think you should make that a new question, and I or someone else can help out with it.  

Remember that responding to questions does take time.  If questioners consistently insist on responses to detailed follow-ups, the result will be fewer experts willing to devote the time to answer the questions.  This will diminish the value of the forum for all of us.

I think that's fair.  Don't you agree?

Regards,

Orchid2
0
 

Author Comment

by:annakellers_pg
ID: 1206757
I have not been given a PERL programming memory solution.  I will not willinging give th points when the question has not been answered.  Give me a solutionf or perl.  You gave me a general solution for a complext question.  I did not offer 175 points for a managers answer to a developers questions.
0
 
LVL 5

Expert Comment

by:b2pi
ID: 1206758
Something seems odd here.... Why are you running out of memory (I just don't want to get into the philosophy aspects).

1.) I would find it ugly to receive more than 20k rows from a single query, and you'd better hope your server is fast enough to get it all out within 5 seconds (or you'd hit timeout problems on CGI), but... let's assume you want all the rows out.  If we look at the code, we have to wonder what it is that's eating memory....

First, try moving @row up above that loop (could be an autoallocation problem?)
To do this, add

my(@row);

to the program, above the

print "<TABLE BORDER=2>\n";

(Which I'm assuming is NOT in a loop).  If that doesn't work, just as a debug exercise, comment out the code within the loop.  I'd really like to find out if you can execute the entire thing under those circumstances without a memory error....

0
 

Author Comment

by:annakellers_pg
ID: 1206759
Have b2pi lock the questions so they can be awarded the points.
0
 
LVL 5

Expert Comment

by:b2pi
ID: 1206760
I'm not sure I understand.  Did you find an answer?
0
 

Author Comment

by:annakellers_pg
ID: 1206761
b2pi gave me an answer I gave to my developer ...they said it was fine.  Have b2pi lock the question and i'll give up the points.
0
 
LVL 5

Accepted Solution

by:
b2pi earned 680 total points
ID: 1206762
I'd presume that it was the undeclared @row which was auto-revivifying?
0
 
LVL 5

Expert Comment

by:b2pi
ID: 1206763
Oh, after complaining loudly that you didn't feel the answers you were getting weren't worth anything, thank you so much for expending the time and energy to answer this question, preventing it from autograding.  I'm sure all experts appreciate this just as much as I do.
0
 
LVL 5

Expert Comment

by:b2pi
ID: 1206764
Hmmm, re-reading that, for some reason it sounds quite snide.  Not intended.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I have been pestered over the years to produce and distribute regular data extracts, and often the request have explicitly requested the data be emailed as an Excel attachement; specifically Excel, as it appears: CSV files confuse (no Red or Green h…
Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans
Suggested Courses

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question