Solved

outofmemory exceptions, please help

Posted on 2004-03-30
28
587 Views
Last Modified: 2008-02-01
hey we are running wls70 in a farm environment on redhat linux enterprise and keep getting out of memory exceptions for large data sets from xml (we generate the webpages from the xml file).

The wls group already increased our memory/heap size to 700mb which should be more than enough. It only happens to 1 particular instance when there is a very large dataset.  We were not able to use any of the memory debugger tools such as jprobe since we access our linux server via ssh and there is no graphic ui attached to it.

Here's the latest log after 1 day of use since reboot. If anyone can shed some light on this it will be greatly appreciated. thanks.

[ 8453][memory ] 672.883: GC 356933K->154974K (524288K) in 425.388 ms
[ 8453][memory ] 673.389: GC 165891K->151619K (524288K) in 342.962 ms
[ 8453][memory ] 673.739: GC 151619K->151083K (524288K) in 502.063 ms
[ 8453][memory ] 674.249: GC 151083K->150134K (524288K) in 437.784 ms
[ 8453][memory ] 674.697: GC 150134K->150082K (524288K) in 383.152 ms
[ 8453][memory ] 675.098: GC 150082K->149416K (524288K) in 655.017 ms
[ 8453][memory ] 675.762: GC 149416K->144247K (524288K) in 639.520 ms
[ 8453][memory ] 676.407: GC 144247K->135180K (524288K) in 515.898 ms
[ 8453][memory ] 676.930: GC 135180K->127807K (524288K) in 476.655 ms
[ 8453][memory ] 677.412: GC 127807K->117806K (524288K) in 693.205 ms
[ 8453][memory ] 21373.560: GC 487584K->146378K (524288K) in 484.512 ms


[ 8454][memory ] 21483.893: GC 524288K->117880K (524288K) in 386.723 ms
<Mar 25, 2004 4:36:02 AM EST> <Info> <HTTP> <101047> <[ServletContext(id=154491569,name=ABC,context-path=/ABC)] /*: init>
<Mar 25, 2004 4:36:02 AM EST> <Info> <HTTP> <101047> <[ServletContext(id=154491569,name=ABC,context-path=/ABC)] /*: Using standard I/O>
mon : 3
mon : 3
[ 8454][memory ] 43206.047: GC 499684K->155802K (524288K) in 383.893 ms
mon : 3
mon : 3
mon : 3
mon : 3
mon : 3
[ 8453][memory ] 55175.603: GC 293354K->147537K (524288K) in 287.490 ms
[ 8453][memory ] 55175.899: GC 147537K->141312K (524288K) in 237.305 ms
[ 8453][memory ] 55176.140: GC 141312K->141234K (524288K) in 266.500 ms
[ 8453][memory ] 55176.412: GC 141234K->139276K (524288K) in 313.818 ms
[ 8453][memory ] 55176.730: GC 139276K->136913K (524288K) in 390.718 ms
[ 8453][memory ] 55177.128: GC 136913K->135383K (524288K) in 481.528 ms
[ 8453][memory ] 55177.654: GC 152115K->137978K (524288K) in 242.307 ms
[ 8453][memory ] 55177.901: GC 137978K->137956K (524288K) in 229.091 ms
[ 8453][memory ] 55178.134: GC 137956K->137724K (524288K) in 278.285 ms
[ 8453][memory ] 55178.417: GC 137724K->137298K (524288K) in 282.001 ms
[ 8453][memory ] 55178.703: GC 137298K->136364K (524288K) in 345.572 ms
[ 8453][memory ] 55179.053: GC 136364K->136296K (524288K) in 448.977 ms
[ 8453][memory ] 55179.512: GC 136296K->135054K (524288K) in 666.687 ms
[ 8453][memory ] 55180.269: GC 143692K->138923K (524288K) in 230.474 ms
[ 8453][memory ] 55180.503: GC 138923K->138875K (524288K) in 223.139 ms
[ 8453][memory ] 55180.730: GC 138875K->138649K (524288K) in 298.802 ms
[ 8453][memory ] 55181.032: GC 138649K->138631K (524288K) in 273.352 ms
[ 8453][memory ] 55181.308: GC 138631K->138597K (524288K) in 246.089 ms
[ 8453][memory ] 55181.557: GC 138597K->138534K (524288K) in 252.921 ms
[ 8453][memory ] 55181.813: GC 138534K->138393K (524288K) in 301.996 ms
[ 8453][memory ] 55182.119: GC 138393K->123204K (524288K) in 1554.888 ms
<Mar 25, 2004 9:46:30 AM EST> <Error> <HTTP> <101017> <[ServletContext(id=154491569,name=ABC,context-path=/ABC)] Root cause of ServletException
java.lang.OutOfMemoryError
0
Comment
Question by:gagaliya
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 8
  • 8
  • 6
  • +2
28 Comments
 
LVL 92

Expert Comment

by:objects
ID: 10718225
do you know which code is actually causing the OutOfMemoryError
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 10718317
Is this a DOM or a SAX operation? The former is always relatively much more memory intensive than the latter. You should SAX process wherever possible
0
 
LVL 1

Author Comment

by:gagaliya
ID: 10718413
Hey I did not design this app so i aplogize for being a bit fuzzy on the details. But it is a relatively standard web app hooked to a db and use xml to generate html. I am not sure if it is dom or sax(can you tell from the log i provided below?).  The only suspicious thing(besides outofmemory) from the log are those exceptions below, i am not sure if they have anything at all to do with outofmemory errors. Of course those exception messages stop at the package class so we dont even know which user class called and they are completely random as far as we can tell.  

Out of memory exception happens at various pages/code, the only thing we can determine is it happens when there is a very large xml data set. So for most clients it runs fine indefinitly, then for one particular client with a lot of data it causes outofmemory exceptions after a day or 2 of usage. Again the memory heap is at 700 mb for our wls server, it still should be more than enough to handle the large datasets.  Thanks for your help!!

Start server side stack trace:
java.lang.OutOfMemoryError:
Start server side stack trace:
java.lang.OutOfMemoryError
End  server side stack trace

End  server side stack trace


Start server side stack trace:
java.lang.OutOfMemoryError:
        at org.apache.xalan.serialize.SerializerToXML.indent(I)V(SerializerToXML
.java:2241)
        at org.apache.xalan.serialize.SerializerToHTML.endElement(Ljava.lang.Str
ing;Ljava.lang.String;Ljava.lang.String;)V(SerializerToHTML.java:656)
        at org.apache.xalan.transformer.ResultTreeHandler.endElement(Ljava.lang.
String;Ljava.lang.String;Ljava.lang.String;)V(ResultTreeHandler.java:280)
        at org.apache.xalan.templates.ElemLiteralResult.execute(Lorg.apache.xala
n.transformer.TransformerImpl;Lorg.w3c.dom.Node;Lorg.apache.xml.utils.QName;)V(E
lemLiteralResult.java:643)
        at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(Lo
rg.apache.xalan.templates.ElemTemplateElement;Lorg.w3c.dom.Node;Lorg.apache.xml.
utils.QName;Z)V(TransformerImpl.java:2251)
        at java.lang.Thread.run()V(Unknown Source)
        at java.lang.Thread.startThreadFromVM(Ljava.lang.Thread;)V(Unknown Sourc
e)
---------
java.lang.ArrayIndexOutOfBoundsException
        at org.apache.xalan.serialize.SerializerToXML.accum(C)V(SerializerToXML.
java:1321)
        at org.apache.xalan.serialize.SerializerToXML.outputLineSep()V(Serialize
rToXML.java:195)
        at org.apache.xalan.serialize.SerializerToXML.indent(I)V(SerializerToXML
.java:2241)
        at org.apache.xalan.serialize.SerializerToHTML.endElement(Ljava.lang.Str
ing;Ljava.lang.String;Ljava.lang.String;)V(SerializerToHTML.java:656)
        at org.apache.xalan.transformer.ResultTreeHandler.endElement(Ljava.lang.
String;Ljava.lang.String;Ljava.lang.String;)V(ResultTreeHandler.java:280)
        at org.apache.xalan.transformer.TransformerImpl.run()V(TransformerImpl.java:3070)
        at java.lang.Thread.run()V(Unknown Source)
        at java.lang.Thread.startThreadFromVM(Ljava.lang.Thread;)V(Unknown Source)
---------
java.lang.ArrayIndexOutOfBoundsException
        at org.apache.xalan.serialize.SerializerToHTML.startElement(Ljava.lang.String;Ljava.lang.String;Ljava.lang.String;Lorg.xml.sax.Attributes;)V(Optimized Method)
        at org.apache.xalan.transformer.QueuedStartElement.flush()V(QueuedStartElement.java:357)
        at org.apache.xalan.transformer.ResultTreeHandler.flushPending(I)V(ResultTreeHandler.java:770)
        at org.apache.xalan.transformer.ResultTreeHandler.endElement(Ljava.lang.String;Ljava.lang.String;Ljava.lang.String;)V(ResultTreeHandler.java:279)
        at org.apache.xalan.templates.ElemLiteralResult.execute(Lorg.apache.xalan.transformer.TransformerImpl;Lorg.w3c.dom.Node;Lorg.apache.xml.utils.QName;)V(ElemLiteralResult.java:643)
        at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(Lorg.apache.xalan.templates.ElemTemplateElement;Lorg.w3c.dom.Node;Lorg.apache.xml.utils.QName;Z)V(TransformerImpl.java:2251)
        at org.apache.xalan.transformer.TransformerImpl.applyTemplateToNode(Lorg.apache.xalan.templates.ElemTemplateElement;Lorg.apache.xalan.templates.ElemTemplateElement;Lorg.w3c.dom.Node;Lorg.apache.xml.utils.QName;)Z(TransformerImpl.java:2134)
        at org.apache.xalan.transformer.TransformerImpl.transformNode(Lorg.w3c.dom.Node;)V(TransformerImpl.java:1246)
        at org.apache.xalan.transformer.TransformerImpl.run()V(TransformerImpl.java:3070)
        at java.lang.Thread.run()V(Unknown Source)
        at java.lang.Thread.startThreadFromVM(Ljava.lang.Thread;)V(Unknown Source)
mon : 3
javax.xml.transform.TransformerException
        at org.apache.xalan.transformer.TransformerImpl.transformNode(Lorg.w3c.dom.Node;)V(TransformerImpl.java:1269)
        at org.apache.xalan.transformer.TransformerImpl.run()V(TransformerImpl.java:3070)
        at java.lang.Thread.run()V(Unknown Source)
        at java.lang.Thread.startThreadFromVM(Ljava.lang.Thread;)V(Unknown Source)
---------
java.lang.ArrayIndexOutOfBoundsException
        at org.apache.xalan.serialize.SerializerToHTML.startElement(Ljava.lang.String;Ljava.lang.String;Ljava.lang.String;Lorg.xml.sax.Attributes;)V(Optimized Method)
        at org.apache.xalan.transformer.QueuedStartElement.flush()V(QueuedStartElement.java:357)
        at org.apache.xalan.transformer.ResultTreeHandler.flushPending(I)V(ResultTreeHandler.java:770)
        at org.apache.xalan.transformer.ResultTreeHandler.endElement(Ljava.lang.String;Ljava.lang.String;Ljava.lang.String;)V(ResultTreeHandler.java:279)
        at org.apache.xalan.templates.ElemLiteralResult.execute(Lorg.apache.xalan.transformer.TransformerImpl;Lorg.w3c.dom.Node;Lorg.apache.xml.utils.QName;)V(ElemLiteralResult.java:643)
        at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(Lorg.apache.xalan.templates.ElemTemplateElement;Lorg.w3c.dom.Node;Lorg.apache.xml.utils.QName;Z)V(TransformerImpl.java:2251)
        at org.apache.xalan.transformer.TransformerImpl.applyTemplateToNode(Lorg.apache.xalan.templates.ElemTemplateElement;Lorg.apache.xalan.templates.ElemTemplateElement;Lorg.w3c.dom.Node;Lorg.apache.xml.utils.QName;)Z(TransformerImpl.java:2134)
Start server side stack trace:
java.lang.OutOfMemoryError:
Start server side stack trace:
java.lang.OutOfMemoryError
End  server side stack trace


Start server side stack trace:
java.lang.OutOfMemoryError:
Start server side stack trace:
java.lang.OutOfMemoryError
End  server side stack trace

End  server side stack trace


Start server side stack trace:
java.lang.OutOfMemoryError:
Start server side stack trace:
java.lang.OutOfMemoryError
End  server side stack trace


Start server side stack trace:
java.lang.OutOfMemoryError:
Start server side stack trace:
java.lang.OutOfMemoryError
End  server side stack trace

End  server side stack trace

End  server side stack trace

End  server side stack trace

>
mon : 3
mon : 3
mon : 3
mon : 3
mon : 3
0
Salesforce Made Easy to Use

On-screen guidance at the moment of need enables you & your employees to focus on the core, you can now boost your adoption rates swiftly and simply with one easy tool.

 
LVL 1

Author Comment

by:gagaliya
ID: 10718439
sorry ignore this part, i pasted it wrong from vim. they are not related at all

Start server side stack trace:
java.lang.OutOfMemoryError:  (not related to the line below)
        at org.apache.xalan.serialize.SerializerToXML.indent(I)V(SerializerToXML
.java:2241)  (not related to the out of memory exception above)
0
 
LVL 92

Expert Comment

by:objects
ID: 10718483
Is it just the one XML file you are having problems with?
0
 
LVL 92

Expert Comment

by:objects
ID: 10718586
if you restart the server and then immediately try to load one of the large data sets do you get the exception?
0
 
LVL 2

Expert Comment

by:Moroni24
ID: 10719301
When you say " a very large dataset" how many bytes are you talking? Like CEHJ mentioned, DOM is ver memory intensive because it loads the ENTIRE xml document before doing anything with it.

Howver, since you didn't write the app, I'm assuming you don't access to the source, which will prevent you from switching to sax. Maybe it would be best to par down the dataset if at all possible.

 
0
 
LVL 1

Author Comment

by:gagaliya
ID: 10719880
-Is it just the one XML file you are having problems with?

No, it has nothing to do with the XML file format. The xml file contains different data and format for each load, the outofmemory exception happens when the XML file contains a very large set of data.


-if you restart the server and then immediately try to load one of the large data sets do you get the exception?

No, it loads ok. Outofmemory only happens on large data sets after a period of time. Usually one or two days. This led me to believe it is a memory leak issue not lack of physical memory

-When you say " a very large dataset" how many bytes are you talking? Like CEHJ mentioned, DOM is ver memory intensive because it loads the ENTIRE xml document before doing anything with it.
Howver, since you didn't write the app, I'm assuming you don't access to the source, which will prevent you from switching to sax. Maybe it would be best to par down the dataset if at all possible.

The page(which caused outofmemory) containing both the html code and the entire xml sheet is 3.55 MB.  How do i tell if it is using DOM or SAX?
I do have full access to the source, just not too familiar with some of the details.  Is the switch to sax easy? or does it require a complete rewrite.  Cutting up the data is really not an option at the moment.

thanks again for your time!

gaga




0
 
LVL 92

Assisted Solution

by:objects
objects earned 200 total points
ID: 10719906
> Outofmemory only happens on large data sets after a period of time.

Then it sounds like you have a memory leak somewhere. So switching to SAX won't fix the problem, what you need to do is determine where the leak is occurring.
0
 
LVL 86

Accepted Solution

by:
CEHJ earned 200 total points
ID: 10719997
>>So switching to SAX won't fix the problem

Unless it happens to be the libraries doing the DOM stuff (quite likely)
0
 
LVL 92

Expert Comment

by:objects
ID: 10720019
So according to CEHJ never use DOM :-D
0
 
LVL 9

Expert Comment

by:mmuruganandam
ID: 10720964
Which JDK version are you using.

In JDK 1.4.2 (not sure), there is a memory leak in the StringBuffer.  That might cause this problem.  Use the latest version of JDK, that might help you in this case.


Regards,
Muruga
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 10722668
>>So according to CEHJ never use DOM :-D

Quite right - if it's avoidable

>>In JDK 1.4.2 (not sure), there is a memory leak in the StringBuffer

? Please expand...
0
 
LVL 9

Expert Comment

by:mmuruganandam
ID: 10722685
In JDK itself there is a memory leak in the StringBuffer code.  So they came up with a patch.

I am not quite sure about the JDK version.  But pretty much sure on 1.4.*

Since SAX parsers are using StringBuffer for all the manipulation, there is a high chance of OutOfMemoryError.

0
 
LVL 86

Expert Comment

by:CEHJ
ID: 10722837
>>.  So they came up with a patch.

I haven't been able to find anything on this.
0
 
LVL 9

Assisted Solution

by:mmuruganandam
mmuruganandam earned 100 total points
ID: 10722863
0
 
LVL 9

Expert Comment

by:mmuruganandam
ID: 10722888
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 10722890
Hmm. That's still seems to be in the realms of 'hearsay' unfortunately
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 10722898
(My last was to your last-before-last)
0
 
LVL 9

Expert Comment

by:mmuruganandam
ID: 10722910
it is 1.4.1_x

0
 
LVL 9

Expert Comment

by:mmuruganandam
ID: 10722911
check it out... I am not too sure about it.
0
 
LVL 1

Author Comment

by:gagaliya
ID: 10724203
hey, i know there is a leak the problem is i don't know how to go about fixing it. since the codes are all on server side and we can only access it via a console. I guess my real questions:

1) how do i know which parse i am using dom / sax
2) are there any good memory debugger tools that will work with just telnet, since i dont have any gui to linux.

thanks

gaga
0
 
LVL 92

Expert Comment

by:objects
ID: 10727520
Many profilers allow you to connect to the running process via socket, would this help you?
0
 
LVL 92

Expert Comment

by:objects
ID: 10727615
> That's still seems to be in the realms of 'hearsay' unfortunately

As does your advice to never use DOM.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 10727647
>>As does your advice to never use DOM.

Your comments suggest you haven't read the thread carefully enough
0
 
LVL 1

Author Comment

by:gagaliya
ID: 10732594
thanks, i managed to get jprobe working with exceed. will see what happens
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 10733645
:-)

Well done for finding that link mmuruganandam, that's a nasty little bug, although i haven't
looked for it in any source i'm using. It must be said that StringBuffers don't actually need to
be buggy in order to cause memory problems - quite a few people don't know how to use
them in an efficient way so they can 'leak' memory anyway.

That's just character buffering though. The real leakiness i would guess (i have never looked at the source
of a DOM implementation) would tend to arise from the 'gotchas' that can occur through reference
management and the garbage collector.

0
 
LVL 92

Expert Comment

by:objects
ID: 10735952
>  i managed to get jprobe working with exceed.

yes jprobe is one that supports remote debugging.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Certificat to iSeries KeyStore 1 30
difference between sorce folder and folder in eclipise 3 64
American Express @Work site and Java 4 95
ArrayList Adding/settign data 3 37
Go is an acronym of golang, is a programming language developed Google in 2007. Go is a new language that is mostly in the C family, with significant input from Pascal/Modula/Oberon family. Hence Go arisen as low-level language with fast compilation…
Basic understanding on "OO- Object Orientation" is needed for designing a logical solution to solve a problem. Basic OOAD is a prerequisite for a coder to ensure that they follow the basic design of OO. This would help developers to understand the b…
This tutorial covers a practical example of lazy loading technique and early loading technique in a Singleton Design Pattern.
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.

734 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question