java jvm - exit code 137 (128 + signal number for SIGKILL) on RHEL6
We have a JVM that is stopping with an exit code 137 (128 + signal number for SIGKILL) and no application logs about the stop.
I assume that the JVM would never kill itself, so this must mean that another process/user killed the JVM?
Also is there some root log that might hold details of what killed the JVM process?
I assume that the JVM would never kill itself, so this must mean that another process/user killed the JVM?
Also is there some root log that might hold details of what killed the JVM process?
ASKER
I have read that if the kernel sends a sigkill then it is in the syslog and should be see by using dmesg. But I see no kill or SIGKILL info in that log.
Regardless, my main question is would the JVM issue a SIGKILL to itself? Or is SIGKILL only produced by other process sending SIGKILL to the JVM?
Regardless, my main question is would the JVM issue a SIGKILL to itself? Or is SIGKILL only produced by other process sending SIGKILL to the JVM?
It wouldn't normally kill itself, no
-XX:ErrorFile=<path> . This was added starting java 5 to specify the path for the hs_err log file . But not sure if a SIGKILL would create a log . You can try that JVM arg and see if a log gets created at the specified path
It wouldn't normally kill itself, noIn fact, afaik, it's not possible for a process to kill itself
Did you eliminate the OOM killer possibility?
ASKER
Not sure if OOM Killer is logged, but I will look into it.
I was using -XX:OnOutOfMemoryError and -XX:OnError and got nothing, also STDERR and STDOUT are logged. And log4j shows no errors, just was working fine and then cuts off without any exceptions.
Since there was nothing in the dmesg log, I am thinking some admin process or person killed it and not the kernel.
I was using -XX:OnOutOfMemoryError and -XX:OnError and got nothing, also STDERR and STDOUT are logged. And log4j shows no errors, just was working fine and then cuts off without any exceptions.
Since there was nothing in the dmesg log, I am thinking some admin process or person killed it and not the kernel.
ASKER
Well, I have not have not been able to replicate the problem again. The system Admins have stated that their were no logs of a SIGKILL, but I did note that our systems are configured to have kill signals in the audit log. And I found you can search the logs via: ausearch -sc kill . So I am hoping this problem will reoccur and that we will get more audit info on it when it does occur. Until then, I can't note if there is a solution in this thread, but thanks for inputs.
OK
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
No solution has been found yet, but the auditing I have noted should help determine who did the kill.
http://stackoverflow.com/questions/726690/who-killed-my-process-and-why