Solved

FileInputStream.readBytes(Native Method) OutOfMemoryError

Posted on 2006-06-16
38
2,795 Views
Last Modified: 2013-11-23
I keep seeing the following in my application:

Exception in thread "Thread-542" java.lang.OutOfMemoryError
        at java.io.FileInputStream.readBytes(Native Method)
        at java.io.FileInputStream.read(FileInputStream.java:177)
        .....

What's FileInputStream doing allocating memory from the heap during a read?
0
Comment
Question by:rstaveley
  • 19
  • 10
  • 7
  • +2
38 Comments
 
LVL 86

Expert Comment

by:CEHJ
ID: 16919723
How are you reading? Please post code
0
 
LVL 24

Accepted Solution

by:
sciuriware earned 40 total points
ID: 16919854
You may assume that all those methods don't waste memory
as they're so much used.

;JOOP!
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16919888
Sorry if this is a bit verbose

--------8<--------
      /*
       * Create a temporary file from the input stream. Unindexable text is stripped out.
       * This includes:
       *
       *      (1) UUEncoded blocks
       *      [Not included... (2) Long lines TODO: should we also strip excessively long tokens]
       *
       * @param is            {@link InputStream} to convert
       * @throws            <code>DocumentHandlerException</code>
       * @return            <code>boolean</code> success/fail
       */
      protected boolean createStrippedTempFileViaLocalBuffering(InputStream is) throws StandardDocumentHandlerException {

            // Sanity check
            if (is == null) {
                  System.out.println(new TimeStamp().toString()+getClass().getName()+": Error: Null InputStream");
                  return false;
            }

            boolean skip_uue = false;

            try {

                  // Create temp file.
                  temp = File.createTempFile("PlainTextHandler",".txt");

                  // Delete temp file when program exits.
                  temp.deleteOnExit();

                  // Write to temp file
                  OutputStream os = new FileOutputStream(temp);

//System.out.println(new TimeStamp().toString()+getClass().getName()+": Opened OutputStream "+(os!=null?"OK":"NULL!!"));

                  if (os == null)
                          return false;

                  byte[] ibuf = new byte[1*1024*1024];      // The InputStream is read into this buffer
                  byte[] obuf = new byte[1*1024*1024];      // The OutputStream writes from this buffer

                  boolean skippingUUE = false;      // This flag is true, while we are processing a UUEncoded block
                  int line_number = 1;            // Current line number
                  int uue_line_number = 0;      // UUEncoded text line number
                  boolean gotNewLineChar = true;      // When we start processing we are on a new line
                  int newLineCharOffset = -1;      // Offset of newLineChar
                  //int lastNewLineCharOffset = -1;      // Offset of last newLineChar
                  int bytesRead;                  // Bytes read from the InputStream
                  long totalBytesRead = 0L;
                  long totalBytesWritten = 0L;

                  while ((bytesRead = is.read(ibuf)) != -1) {

                        int unskippedDataOffset = 0;      // Marks the start of the data, which shouldn't be skipped

                        // Increase the total
                        totalBytesRead += bytesRead;

//System.out.println(new TimeStamp().toString()+getClass().getName()+": Processed "+totalBytesRead+" bytes");

                        int offset = 0;                  // Offset into the input buffer
                        int obufOccupiedSize = 0;      // Amount of the output buffer now loaded

                        // Walk through the input buffer
                        while (offset < bytesRead) {

                              // If we have not already found a newLineChar, we should look for it
                              if (!gotNewLineChar) {

                                    // Look for the newLineChar for the start/end of UUEncoding
                                    while (offset < bytesRead) {
                                          if (ibuf[offset] == newLineChar) {
                                                gotNewLineChar = true;
                                                newLineCharOffset = offset;
                                                break;
                                          }
                                          ++offset;
                                    }

                                    // If we still haven't got a newLineChar, we must have reached the
                                    // end if the input buffer, without finding one.
                                    if (!gotNewLineChar)
                                          break;

                                    // Text file line number
                                    ++line_number;

                                    // UUEncoded text line number
                                    if (skippingUUE)
                                          ++uue_line_number;

                              }

                              // We are on a new line, if control arrives here.

                              // Special handling for situation where last character read is the
                              // newLineCharacter. We mark the newLineCharOffset as -1 and use
                              // the same logic that applies for the 1st line.
                              if (newLineCharOffset == bytesRead-1) {
                                    newLineCharOffset = -1;            // Analogous to 1st line
                                    break;
                              }

                              // If we are not in a UUEncoded block, we should look for the start
                              // of one. The signature that we look for is 10 characters long. If the
                              // signature straddles buffers, we do not bother with it. This is a
                              // performance optimisation, which we may need to reconsider, but if
                              // the buffer is large enough the probability is reasonably slim
                              if (!skippingUUE) {

                                    if (      newLineCharOffset+10 < bytesRead
                                          && ibuf[newLineCharOffset+1] == 'b'
                                          && ibuf[newLineCharOffset+2] == 'e'
                                          && ibuf[newLineCharOffset+3] == 'g'
                                          && ibuf[newLineCharOffset+4] == 'i'
                                          && ibuf[newLineCharOffset+5] == 'n'
                                          && ibuf[newLineCharOffset+6] == ' '
                                          && ibuf[newLineCharOffset+7] >= '0' && ibuf[newLineCharOffset+7] <= '8'
                                          && ibuf[newLineCharOffset+8] >= '0' && ibuf[newLineCharOffset+8] <= '8'
                                          && ibuf[newLineCharOffset+9] >= '0' && ibuf[newLineCharOffset+9] <= '8'
                                          && ibuf[newLineCharOffset+10] >= ' '
                                    ) {

                                          // Walk past the next 10 characters
                                          offset = newLineCharOffset+10;

                                          // We should skip the UUEncoded block now
                                          skippingUUE = true;

                                          // Copy everything from the unskippedDataOffset
                                          // up to the newLineCharOffset
                                          // into the destination buffer

                                          if (newLineCharOffset > unskippedDataOffset) {
                                                System.arraycopy(
                                                      ibuf,unskippedDataOffset
                                                      ,obuf,obufOccupiedSize
                                                      ,newLineCharOffset-unskippedDataOffset
                                                );
                                                obufOccupiedSize += newLineCharOffset-unskippedDataOffset;
                                          }
                                    }

                              //      // TODO: See if this is a long line. We could strip long lines??
                              //      else {
                              //      }

                                    // We'll move
                                    gotNewLineChar = false;

                              }

                              // We are skipping
                              else {

                                    // We are handling the newLineChar now. Let's unset the flag up front.
                                    gotNewLineChar = false;

                                    // Look for the end of UUEncoded block skipping
                                    if (      newLineCharOffset+10 < bytesRead
                                          && ibuf[newLineCharOffset+1] == 'e'
                                          && ibuf[newLineCharOffset+2] == 'n'
                                          && ibuf[newLineCharOffset+3] == 'd'
                                    ) {

                                          // Walk past the next 3 characters
                                          offset = newLineCharOffset+3;

                                          // Flag the fact that we are no longer
                                          skippingUUE = false;

                                          // Look for the newLineChar after end
                                          while (offset < bytesRead) {
                                                if (ibuf[offset] == newLineChar) {
                                                      gotNewLineChar = true;
                                                      newLineCharOffset = offset;
                                                      unskippedDataOffset = offset+1;
                                                      break;
                                                }
                                                ++offset;
                                          }

                                          // Infrequent condition not found newLineChar after end
                                          if (!gotNewLineChar) {
                                                unskippedDataOffset = bytesRead;
                                                break;
                                          }
                                    }
                              }

                              // Keep a note of this offset - we want to check for long lines
                              //lastNewLineCharOffset = newLineCharOffset;

                              // Move off the newLineChar to the next character
                              ++offset;

                        } // Walk through input buffer

                        // If we are not currently skipping, we should copy the remaining
                        // data into the output buffer.
                        if (!skippingUUE && bytesRead > unskippedDataOffset) {

                              // Sneaky optimisation - if the entire read is due
                              // to be copied to the output buffer, let's take it
                              // directly from the input buffer
                              if (obufOccupiedSize == 0) {
                                    os.write(ibuf,unskippedDataOffset,bytesRead-unskippedDataOffset);
                                    if ((totalBytesWritten += bytesRead-unskippedDataOffset) >= truncatedFileSize) {
System.out.println(new TimeStamp().toString()+getClass().getName()+": Truncated PlainText input ("+totalBytesWritten+" bytes)");
                                          break;
                                    }
                                    continue;
                              }

                              // Remaining data starts at unskippedDataOffset
                              System.arraycopy(
                                    ibuf,unskippedDataOffset
                                    ,obuf,obufOccupiedSize
                                    ,bytesRead-unskippedDataOffset
                              );
                              obufOccupiedSize += bytesRead-unskippedDataOffset;
                        }

                        // Write any data loaded into the output buffer to the stream
                        if (obufOccupiedSize > 0) {
                              os.write(obuf,0,obufOccupiedSize);
                              if ((totalBytesWritten += bytesRead-unskippedDataOffset) >= truncatedFileSize) {
System.out.println(new TimeStamp().toString()+getClass().getName()+": Truncated PlainText input ("+totalBytesWritten+" bytes)");
                                    break;
                              }
                        }


                  } // Loading input buffer from InputStream

                  is.close();
                  os.close();

                  // Show how many UUEncoded lines we've skipped
                  if (uue_line_number > 0) {
System.out.println(new TimeStamp().toString()+getClass().getName()+": Skipped "+uue_line_number+" lines of UUEncoded text in "+line_number+" lines ("+totalBytesRead+" bytes)");
                  }
                  else {
System.out.println(new TimeStamp().toString()+getClass().getName()+": "+line_number+" lines processed ("+totalBytesRead+" bytes - no UUE)");
                  }

            }
            catch (IOException e) {
//System.out.println(new TimeStamp().toString()+getClass().getName()+": IOException "+e.toString());
                  throw new StandardDocumentHandlerException("Cannot read the text document",e);
            }
            catch (Exception e) {
//System.out.println(new TimeStamp().toString()+getClass().getName()+": Exception "+e.toString());
                  throw new StandardDocumentHandlerException("Exception caught in PlainTextHandler",e);
            }
            return true;
      }
--------8<--------

It is supposed to be a quick way of stripping UUEncoded text from plain text.
0
 
LVL 3

Assisted Solution

by:Kanti
Kanti earned 40 total points
ID: 16919933
You can override this default using the -Xms command-line option
 using -Xms256m -Xmx1024m
http://java.sun.com/j2se/1.5.0/docs/guide/vm/gc-ergonomics.html
0
 
LVL 86

Assisted Solution

by:CEHJ
CEHJ earned 130 total points
ID: 16919944
Can't see anything immediately and have to go out now. Close those streams in a finally block. Are you calling that method in a loop?
0
 
LVL 30

Assisted Solution

by:mayankeagle
mayankeagle earned 40 total points
ID: 16920028
Try streaming the reading by using int read ( byte[], int, int ) instead of int read ().

Also:

>> os.write(obuf,0,obufOccupiedSize);

Call os.flush () after that.
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16920181
Kanti, it is already running with -Xmx1024m. It is a server application. It is a daemon processing TCP/IP requests. I wonder if -XX:+UseParallelGC is what I need?

CEHJ, I'll do as you suggest in the finally block. It isn't called in a loop, but it typically gets called 5 times a second in my application as and when a file needs "stripping".

mayankeagle, it is read (byte[]) . I'll try explicitly flushing the FileOutputStream, but it isn't a BufferedOutputStream, so should I really expect a reduction in heap usage doing that?
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16920202
... How can I tell if it is being treated as a "Server-Class Machine"?
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 16920246
>> it is read (byte[])

Yeah, try using the other one. But before that, try the flush () [also try making it a buffered output-stream if you want)]
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16920313
> also try making it a buffered output-stream if you want

Wouldn't that increase the heap requirement?
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 16920334
If you keep flushing, no. But anyway, don't bother about it - I suggested it because that is the one I normally use. First, just try with the flush ().
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16920478
mayankeagle,

>> it is read (byte[])
> Yeah, try using the other one

According to http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html#read%28byte%5B%5C%29 the two are equivalent.
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 16920504
Nopes, in the other one you can specify how many bytes you want to read. In this one (read (byte[])), it always reads till array.length
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16920508
Bah, I wish EE could handle [ and ( itself in URLs. I'll try that again.

According to http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html#read%28byte%5c%5d%29 the two are equivalent.
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 16920511
Anyway what is to be tried first is the flush ()
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16920517
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16920719
I watching my heap slowly getting used up in my application. It may be that my problem is elsewhere in the application and the fact that createStrippedTempFileViaLocalBuffering gets called often is why we see the lack of memory, when memory is scarce.

Is there a heap inspector tool which I could use on a headless server to see where memory is being used up in the application?

Ominously, I note that jstack is having problems stack walking, long before the OutOfMemory manifests itself.

--------8<--------
keystone@gse-mta-10:~$ jstack 30267
Attaching to process ID 30267, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 1.5.0_06-b05
Thread 30558: (state = IN_NATIVE)
 - java.net.SocketInputStream.socketRead0(java.io.FileDescriptor, byte[], int, int, int) @bci=0 (Compiled frame; information may be imprecise)
 - java.net.SocketInputStream.read(byte[], int, int) @bci=84, line=129 (Compiled frame)
 - java.io.BufferedInputStream.fill() @bci=175, line=218 (Compiled frame)
 - com.emailsystems.keystone.BasicIndexerServer.run() @bci=39, line=408 (Compiled frame)
Error occurred during stack walking:
java.lang.NullPointerException
        at sun.jvm.hotspot.runtime.Frame.addressOfStackSlot(Frame.java:214)
        at sun.jvm.hotspot.runtime.x86.X86Frame.addressOfInterpreterFrameMethod(X86Frame.java:442)
        at sun.jvm.hotspot.runtime.Frame.getInterpreterFrameMethod(Frame.java:354)
        at sun.jvm.hotspot.runtime.InterpretedVFrame.getMethod(InterpretedVFrame.java:19)
        at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:51)
        at sun.jvm.hotspot.tools.JStack.run(JStack.java:41)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:204)
        at sun.jvm.hotspot.tools.JStack.main(JStack.java:58)


Thread 30289: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 - com.emailsystems.keystone.MultiIndexIndexerServer$1.run() @bci=7, line=395 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)


Thread 30282: (state = BLOCKED)


Thread 30277: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=116 (Compiled frame)
Error occurred during stack walking:
java.lang.NullPointerException
        at sun.jvm.hotspot.runtime.Frame.addressOfStackSlot(Frame.java:214)
        at sun.jvm.hotspot.runtime.x86.X86Frame.getEntryFrameCallWrapper(X86Frame.java:452)
        at sun.jvm.hotspot.runtime.Frame.entryFrameIsFirst(Frame.java:379)
        at sun.jvm.hotspot.runtime.Frame.isFirstFrame(Frame.java:154)
        at sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:109)
        at sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:134)
        at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:50)
        at sun.jvm.hotspot.tools.JStack.run(JStack.java:41)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:204)
        at sun.jvm.hotspot.tools.JStack.main(JStack.java:58)


Thread 30276: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=474 (Compiled frame)
Error occurred during stack walking:
java.lang.NullPointerException
        at sun.jvm.hotspot.runtime.Frame.addressOfStackSlot(Frame.java:214)
        at sun.jvm.hotspot.runtime.x86.X86Frame.getEntryFrameCallWrapper(X86Frame.java:452)
        at sun.jvm.hotspot.runtime.Frame.entryFrameIsFirst(Frame.java:379)
        at sun.jvm.hotspot.runtime.Frame.isFirstFrame(Frame.java:154)
        at sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:109)
        at sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:134)
        at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:50)
        at sun.jvm.hotspot.tools.JStack.run(JStack.java:41)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:204)
        at sun.jvm.hotspot.tools.JStack.main(JStack.java:58)


Thread 30267: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)
 - java.net.PlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=384 (Interpreted frame)
 - java.net.ServerSocket.implAccept(java.net.Socket) @bci=50, line=450 (Interpreted frame)
 - java.net.ServerSocket.accept() @bci=48, line=421 (Interpreted frame)
 - com.emailsystems.keystone.BasicIndexerServer.main(java.lang.String[]) @bci=624, line=300 (Interpreted frame)
--------8<--------
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 16920751
>> java.net.ServerSocket.accept() @bci=48, line=421 (Interpreted frame)
>> - com.emailsystems.keystone.BasicIndexerServer.main(java.lang.String[])

Perhaps you are getting too many client socket connections.
0
 
LVL 86

Assisted Solution

by:CEHJ
CEHJ earned 130 total points
ID: 16922377
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 17

Author Comment

by:rstaveley
ID: 16925959
I'm running the application on a headless server, and am trying an appraisal version of JProfiler through an SSH tunnel (export DISPLAY=127.0.0.1:0.0, tunneling 6000:127.0.0.1:6000). Viewing JProfiler on Cygwin/X takes several minutes to display its windows. It is if anything worse with XWin32 than Cygwin/X. It is too sluggish to get any valuable information from it. I realise that this isn't an optimum set-up, but it seems like I'm struggling to get an unwanted GUI up and running. I'm got adverse to adding code to my application. Is there something I could add to it that I could call, which would dump a snap shot of the heap at that instant? Or is JProfiler the best tool for the job?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16925986
I didn't realise you were running under those circumstances. Not sure about non-GUI profilers. If your box is *nix you might just try top ;-)
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16926007
Alas, top wouldn't tell me which classes are guzzling the heap :-)

I'll do a bit of Googling to see if I can find something text-based heap inspector. It is too bad jstack doesn't do it.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16926010
>>Alas, top wouldn't tell me which classes are guzzling the heap :-)

Sorry - got a bit distracted then - of course it wouldn't ;-)
0
 
LVL 86

Assisted Solution

by:CEHJ
CEHJ earned 130 total points
ID: 16926017
Try some of these

http://tinyurl.com/eaxds
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16926066
Yikes: Searching projects gives 14008 results!

I'll do that though, Charles, and maybe have a go with -XX:+HeapDumpOnOutOfMemoryError too, which I see at http://blogs.sun.com/roller/page/alanb?entry=heap_dumps_are_back_with. Could HPROF be what I need?

Would you expect jmap to be able to dump details about classes loded in the heap? I'm seeing "java.lang.NullPointerException during traversal", when I use jmap with -permstat, but I confess I don't really know what I'm doing - it doesn't seem to support the dump option in my installation.

--------8<--------
keystone@thb-mta-10:~$ jmap -permstat 24174
Attaching to process ID 24174, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 1.5.0_06-b05
finding class loader instances ..done.
computing per loader stat ..done.
please wait.. computing liveness..LivenessAnalysis: WARNING: java.lang.NullPointerException during traversal
LivenessAnalysis: WARNING: java.lang.NullPointerException during traversal
done.
class_loader    classes bytes   parent_loader   alive?  type

<bootstrap>     884     2004776   null          live    <internal>
0x49fe2fe8      1       1416    0x49bc0060      dead    sun/reflect/DelegatingClassLoader@0x45c02898
0x49bc5290      1       1416      null          dead    sun/reflect/DelegatingClassLoader@0x45c02898
0x4a0d2fd8      1       1416    0x49bc0060      dead    sun/reflect/DelegatingClassLoader@0x45c02898
   .... <lots of these!!> ....
0x4a0cf860      1       1416    0x49bc0060      dead    sun/reflect/DelegatingClassLoader@0x45c02898

total = 120     1875    4295040     N/A         alive=3, dead=117           N/A
--------8<--------
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16926111
I haven't used jmap so you're ahead of me there
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16926224
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16926332
yes - keep us posted ;-)
0
 
LVL 17

Author Comment

by:rstaveley
ID: 16993070
Apologies for keeping this question open so long. I've been running around like a healess chicken for the last 10 days.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16996195
No problem
0
 
LVL 30

Expert Comment

by:mayankeagle
ID: 16999537
Get well soon ;)
0
 
LVL 17

Author Comment

by:rstaveley
ID: 17062781
Update...

Initial impressions of JHAT+JMAP are that it ought to be perfect for my purposes because I can snap-shot a map whenever I want, but the file generated by JMAP so far has always generated messages like:

C:\Program Files\Java\jdk1.6.0\bin>jhat -stack false "C:\Documents and Settings\Rob Staveley\Desktop\indexd.jmap"
Reading from C:\Documents and Settings\Rob Staveley\Desktop\indexd.jmap...
Dump file created Fri Jul 07 21:42:44 BST 2006
java.io.IOException: Unrecognized heap dump sub-record type:  16
        at com.sun.tools.hat.internal.parser.HprofReader.readHeapDump(HprofReader.java:485)
        at com.sun.tools.hat.internal.parser.HprofReader.read(HprofReader.java:222)
        at com.sun.tools.hat.internal.parser.Reader.readFile(Reader.java:79)
        at com.sun.tools.hat.Main.main(Main.java:143)

I've been generating the map file from Sun 1.5, but JHAT is bundled with Sun 1.6. I've tried using HAT from https://hat.dev.java.net/ on the same JVM (and the same Linux server) for what it is worth too, but got the same result. It looks like others have had the same problem generating map files - see http://www.lshift.net/blog/2006/03/08/java-memory-profiling-with-jmap-and-jhat.

Looks like I've got to try and generate an HPROF file, using -Xrunhprof:file=/tmp/indexd.hprof,format=b, but that's not something that I really wanted to do, because it is a daemon. Aw well.
0
 
LVL 17

Author Comment

by:rstaveley
ID: 17064576
HPROF is no good for me, because I need to be able to kill my application with SIGTERM to get the dump, and my application doesn't seem to respond to SIGTERM. I can only kill it with SIGKILL which of course prevents the -Xrunhprof:file=/tmp/indexd.hprof,format=b file from being generated.

I guess I need to get to the bottom of why SIGTERM is getting trapped - I've not installed any ShutdownHooks, but I guess I need to trawl through all the jars I'm using to see if one is installed. I've just created another question for that - see http:Q_21912406.html.
0
 
LVL 17

Author Comment

by:rstaveley
ID: 17089407
I ought to wind up this question now, though it hasn't been very conclusive.

The fact that JSTACK is having problems walking through the stack and that the daemon isn't responding to SIGTERM makes me think that I'm probably getting a sick virtual machine.

I'm seeing this sort of thing sometimes too:

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  SIGSEGV (0xb) at pc=0x400ee5b6, pid=28684, tid=38158352
#
# Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
# Problematic frame:
# C  [libc.so.6+0x7b5b6]  strcpy+0x26
#
# An error report file with more information is saved as hs_err_pid28684.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
0
 
LVL 17

Author Comment

by:rstaveley
ID: 17090525
Interesting to see that the SIGSEGVs are all on the same strcpy+0x26 and same program counter (ps) every time:

#  SIGSEGV (0xb) at pc=0x400ee5b6, pid=11289, tid=4390928
#  SIGSEGV (0xb) at pc=0x400ee5b6, pid=14428, tid=4980752
#  SIGSEGV (0xb) at pc=0x400ee5b6, pid=14777, tid=4784144
0
 
LVL 17

Author Comment

by:rstaveley
ID: 17090603
I should have looked at the generated log files:

C  [libc.so.6+0x7b5b6]  strcpy+0x26
C  [libjava.so+0xf644]
C  [libjava.so+0x1671d]  Java_java_io_UnixFileSystem_deleteOnExit+0x7d
j  java.io.UnixFileSystem.deleteOnExit(Ljava/io/File;)Z+0
j  java.io.File.deleteOnExit()V+20

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.io.UnixFileSystem.deleteOnExit(Ljava/io/File;)Z+0
j  java.io.File.deleteOnExit()V+20

It looks like deleteOnExit is my culprit - i.e. http://java.sun.com/j2se/1.5.0/docs/api/java/io/File.html#deleteOnExit%28%29 - I expect this is getting called when the file is already deleted. I'll avoid it and see if peace is restored to the world.
0
 
LVL 17

Author Comment

by:rstaveley
ID: 17101909
Time to put this question to bed. My error was using deleteOnExit in a daemon, which doesn't exit for ages. It wasn't handled too well by the virtual machine. Thanks to you all!
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 17103268
:-)
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

An old method to applying the Singleton pattern in your Java code is to check if a static instance, defined in the same class that needs to be instantiated once and only once, is null and then create a new instance; otherwise, the pre-existing insta…
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Video by: Michael
Viewers learn about how to reduce the potential repetitiveness of coding in main by developing methods to perform specific tasks for their program. Additionally, objects are introduced for the purpose of learning how to call methods in Java. Define …
The viewer will learn how to implement Singleton Design Pattern in Java.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now