Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

JBOSS crashes

Posted on 2014-03-13
13
Medium Priority
?
1,132 Views
Last Modified: 2014-03-17
Dear Gurus,

I am getting following error

2014-03-13 15:21:19,101 INFO  [STDOUT] (Thread-1) Posting Shutdown Request to the server...

2014-03-13 15:21:19,101 INFO  [org.jboss.bootstrap.impl.base.server.AbstractServer] (Thread-22) Stopping: JBossAS [6.1.0.Final "Neo"]
2014-03-13 15:21:19,117 INFO  [org.apache.coyote.http11.Http11Protocol] (Thread-22) Pausing Coyote HTTP/1.1 on http-0.0.0.0-8080
2014-03-13 15:21:19,117 INFO  [org.apache.coyote.http11.Http11Protocol] (Thread-22) Stopping Coyote HTTP/1.1 on http-0.0.0.0-8080
2014-03-13 15:21:19,132 INFO  [org.apache.coyote.ajp.AjpProtocol] (Thread-22) Pausing Coyote AJP/1.3 on ajp-0.0.0.0-8009
2014-03-13 15:21:19,132 INFO  [org.apache.coyote.ajp.AjpProtocol] (Thread-22) Stopping Coyote AJP/1.3 on ajp-0.0.0.0-8009
2014-03-13 15:24:30,902 INFO  [org.apache.coyote.http11.Http11Protocol] (Thread-2) Starting Coyote HTTP/1.1 on http-0.0.0.0-8080
2014-03-13 15:24:30,917 INFO  [org.apache.coyote.ajp.AjpProtocol] (Thread-2) Starting Coyote AJP/1.3 on ajp-0.0.0.0-8009
2014-03-13 15:24:30,917 INFO  [org.jboss.bootstrap.impl.base.server.AbstractServer] (Thread-2) JBossAS [6.1.0.Final "Neo"] Started in 20s:467ms
2014-03-13 15:26:34,593 INFO  [org.jboss.system.server.jmx.JMXKernel] (Thread-16) Server exit called, exiting the JVM now!

JBOSS 6.1.0
Operating System = Windows 2008 Enterprise Edition R2

THIS IS PRODUCTION SERVER AND IT GET CRASHED CAN YOU PLEASE HELP ME TO IDENTIFY WHY THIS SERVER GET CRASHED

Thanks
Netsol-NOS
0
Comment
Question by:Netsol-NOS
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 7
  • 6
13 Comments
 
LVL 86

Expert Comment

by:CEHJ
ID: 39928636
I am getting following error
There is no error shown there
0
 

Author Comment

by:Netsol-NOS
ID: 39928662
Dear Cehj,

Thank you for your reply.

I have increased the following settings:

FROM:
rem # JVM memory allocation pool parameters - modify as appropriate.
set "JAVA_OPTS=-Xms128M -Xmx512M -XX:MaxPermSize=256M"

TO:
rem # JVM memory allocation pool parameters - modify as appropriate.
set "JAVA_OPTS=-Xms256M -Xmx1024M -XX:MaxPermSize=512M"


Please note that when i check the task manager i realized that the java (JDK) is 32 bit and it is running on 64 bit OS.

So what do you suggest should i change the JDK into 64 bit ?

Secondly in 2 hours the log file get 20 + GB so i stopped the PRODUCTION Server and copy the JBOSS folder into D drive and then again start the server.
After starting the behavior is same the JBOSS Black Console is continuously running like FLASH and the log file is continuously increasing.

After crashed in morning i restart the server till now the server get crashed 2 times.

Please suggest i am in very big trouble.

Thanks
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 39928769
I've still to see any evidence of the problem. You need to post the ERROR trace from the log
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 

Author Comment

by:Netsol-NOS
ID: 39928786
2014-03-13 22:30:03,963 INFO  [STDOUT] (Thread-1) Posting Shutdown Request to the server...

2014-03-13 22:30:03,963 INFO  [org.jboss.bootstrap.impl.base.server.AbstractServer] (Thread-24) Stopping: JBossAS [6.1.0.Final "Neo"]
2014-03-13 22:30:03,963 INFO  [org.apache.coyote.http11.Http11Protocol] (Thread-24) Pausing Coyote HTTP/1.1 on http-0.0.0.0-8080
2014-03-13 22:30:03,963 INFO  [org.apache.coyote.http11.Http11Protocol] (Thread-24) Stopping Coyote HTTP/1.1 on http-0.0.0.0-8080
2014-03-13 22:30:03,994 INFO  [org.apache.coyote.ajp.AjpProtocol] (Thread-24) Pausing Coyote AJP/1.3 on ajp-0.0.0.0-8009
2014-03-13 22:30:03,994 INFO  [org.apache.coyote.ajp.AjpProtocol] (Thread-24) Stopping Coyote AJP/1.3 on ajp-0.0.0.0-8009
2014-03-14 11:31:45,984 INFO  [org.apache.coyote.http11.Http11Protocol] (Thread-2) Starting Coyote HTTP/1.1 on http-0.0.0.0-8080
2014-03-14 11:31:45,984 INFO  [org.apache.coyote.ajp.AjpProtocol] (Thread-2) Starting Coyote AJP/1.3 on ajp-0.0.0.0-8009
2014-03-14 11:31:45,984 INFO  [org.jboss.bootstrap.impl.base.server.AbstractServer] (Thread-2) JBossAS [6.1.0.Final "Neo"] Started in 22s:136ms

2014-03-14 11:33:26,025 ERROR
[org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/das].[jsp]] (http-0.0.0.0-8080-6) Servlet.service() for servlet jsp threw exception: java.lang.StackOverflowError
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:225) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at
org.apache.catalina.core.ApplicationHttpRequest.getAttribute(ApplicationHttpRequest.java:227) [:6.1.0.Final]
      at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:216) [:6.1.0.Final]
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [:1.0.0.Final]
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:324) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:242) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:734) [:6.1.0.Final]
      at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:541) [:6.1.0.Final]
      at
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 39928938
Looks like one of your filtered jsp servlets is causing some stack overflow. Maybe a loop or recursion failing to exit
0
 

Author Comment

by:Netsol-NOS
ID: 39929137
Now problem is not this..

Problem is Log files are continuously producing and JBOSS console (which is black window on desktop) continuously flashing very fast with above data which i mentioned.

Hence the log file get filled in 2 hours almost 20 GB

how can i stop producing this much log file (20 GB log)
0
 
LVL 86

Accepted Solution

by:
CEHJ earned 2000 total points
ID: 39929517
0
 

Author Comment

by:Netsol-NOS
ID: 39929627
thank you let me check
0
 

Author Closing Comment

by:Netsol-NOS
ID: 39933642
Dear All,

We have performed some actions in the jboss-logging.xml and then the problem has been resolved.

Thanks
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 39933659
So what was causing the crash?
0
 

Author Comment

by:Netsol-NOS
ID: 39933667
Right now dont know the reason but may be following reasons.

11:12:42,600 INFO  [STDOUT] java.sql.SQLRecoverableException: Io ¿¿: Connecti
n reset by peer: socket write error
11:19:24,568 INFO  [STDOUT] in setting procedure of querynull
11:19:31,978 INFO  [STDOUT] java.sql.SQLRecoverableException: Io ¿¿: Connecti
n reset by peer: socket write error
11:19:39,825 INFO  [STDOUT] in setting procedure of querynull
11:19:54,005 INFO  [STDOUT] java.sql.SQLRecoverableException: Io ¿¿: Connecti
n reset by peer: socket write error
11:39:35,994 INFO  [STDOUT] java.sql.SQLRecoverableException: Io ¿¿: Connecti
n reset by peer: socket write error
11:53:34,125 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localho
t].[/das].[jsp]] Servlet.service() for servlet jsp threw exception: java.lang.O
tOfMemoryError: Java heap space

We have changed following things.

Xms has been increased to 512
Xmx has been increased to 2048
Maxpermsize has been increased to 1024



Shifted JAVA from 32 bit to 64 bit (JDK 1.6 64 bit)

Limit the log files and now JAVA is taking right now 2.3 GB

Thanks
0
 

Author Comment

by:Netsol-NOS
ID: 39933670
Can you please guide me

I have Windows 2008 R2 64 bit with 64 gb memory.

JBOSS is installed with JDK 1.6 64 bit.

I have set following JAVA HEAP SETTINGS.

Xms has been increased to 512
Xmx has been increased to 2048
Maxpermsize has been increased to 1024


My Task Manager shows the Java is now taking 1.9 GB only java.

Please suggest what will be the next step should we increase the following to 4 GB

Xmx has been increased to 2048 to 4096 or not.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 39933817
What you need to do first is to profile it and find out what is taking up all the memory. If there's a fundamental flaw, adjusting memory parameters isn't going to cure the problem necessarily
0

Featured Post

Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

An old method to applying the Singleton pattern in your Java code is to check if a static instance, defined in the same class that needs to be instantiated once and only once, is null and then create a new instance; otherwise, the pre-existing insta…
For beginner Java programmers or at least those new to the Eclipse IDE, the following tutorial will show some (four) ways in which you can import your Java projects to your Eclipse workbench. Introduction While learning Java can be done with…
Viewers will learn about the regular for loop in Java and how to use it. Definition: Break the for loop down into 3 parts: Syntax when using for loops: Example using a for loop:
This tutorial will introduce the viewer to VisualVM for the Java platform application. This video explains an example program and covers the Overview, Monitor, and Heap Dump tabs.
Suggested Courses

610 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question