asked on

Tomcat 8.0.15 Application randomly freezing.

I have 3 win 2012 R2 servers on vmware with same version of tomcat 8 and same application.
My Dev and Stage are loading fine and dont crash.
But my new prod server which isn't live yet keeps freezing.

Ive ruled out any network or firewall issue. I have another server on same subnet and it wont load the page either.

When the application freezes the only way to unfreeze is it restart tomcat or load the page from localhost server browser. As soon as I do that all other browser sessions load right away.

When its working its working great everywhere, but its freezing every few min.

My old server was tomcat 6 and jvm had 1GB ram alocated, the new server has 2GB.
CPU and memory usage are low so its not a server performance thing.

I do have this error in the logs but it also shows on dev and stage servers too, but they dont have any issues.

02-Feb-2015 10:56:57.559 WARNING [PUBLIC-IP-startStop-2] org.apache.catalina.session.StandardSession.doWriteObject Cannot serialize session attribute domain.com.controller.staff.report.SearchInstitutionProfiles.session.table-listing-criteria for session 767775D8E32CF707B21A6AF94D2D013D
 java.io.NotSerializableException: domain.com.controller.staff.report.CriteriaBuilder$Criterion
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
	at java.util.ArrayList.writeObject(ArrayList.java:747)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
	at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
	at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
	at org.apache.catalina.session.StandardSession.doWriteObject(StandardSession.java:1710)
	at org.apache.catalina.session.StandardSession.writeObjectData(StandardSession.java:1116)
	at org.apache.catalina.session.StandardManager.doUnload(StandardManager.java:401)
	at org.apache.catalina.session.StandardManager.unload(StandardManager.java:320)
	at org.apache.catalina.session.StandardManager.stopInternal(StandardManager.java:487)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232)
	at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5409)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232)
	at org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1425)
	at org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1414)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

02-Feb-2015 10:56:57.621 WARNING [PUBLIC-IP-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [apply] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
02-Feb-2015 10:56:57.621 WARNING [PUBLIC-IP-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [apply] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142)
 com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:43)

Open in new window

girionis

You can solve this error by making the class in question Serializable (you need to implement this interface). Not sure if this will help but it's worth a try.

baysysadmin

ASKER

How do I do that. Can you post a link to a article please.
Tomcat is not my strong suit.

Thanks

baysysadmin

ASKER

I am one step closer.
it looks like a SSL issue.
I took out the SSL connector and it loads and stays fine on http.

My stage didnt have a SSL connector so I put one in exactly the way it was on prod, but its not crashing like prod did, not sure why.

girionis

If you have narrowed it down to an SSL issue, I'd say it's a configuration problem now. You have something in the production environment that you don't have on dev. Can you compare the tomcat configuration files? Can you also compare how you start up tomcat, what switches you're using?

baysysadmin

ASKER

Ive been looking but so far they look identical except these items.

Stage is using 6.3 but prod and Dev have 7.4
-Djava.security.auth.login.config=C:\PaymentechSDK_6.3.0\login.config

Other than that all other configs and settings are the same, and versions.

Im out of options, other than reinstalling tomcat completely.

baysysadmin

ASKER

I did a reinstall of tomcat but it didnt help.
I also removed the HTTP connector, and I was able to connect using HTTPS but it was still doing that freezing thing.

I dont think its a connector issue but rather something with SSL. I just cant figure out what.

baysysadmin

ASKER

Do I need to have Apache installed in order to use Apache Tomcat with SSL?

On my stage servers I have apache 2.4 installed but its used for other websites.
Tomcat is used for 1 specific application.

Thats the only other difference I can think of.

baysysadmin

ASKER

Ive also tried changing the port to 8443 with no luck.
The freezing keeps happening no matter what.

When there is only HTTP connector the freezing goes away.

This is my config

<Connector port="80" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="443" address="XXXXX.135" />

<Connector port="443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="150" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" address="XXXXX.135" keystoreFile="D:\CERTIFICATES\mykeystore" keystorePass="*****" keyAlias="*****" />

girionis

If you use Apache with Tomcat what happens is that Apache serves the static files (HTML, CSSS etc) while Tomcart serves the JSP files and servlets. But if you have Apache in front of Tomcat it is possible that Apache interferes with the SSL requests before it forwards them to Tomcat. IN order to be 100% sure that the problem is with Apache you will need to have the exact same setup with both computers.

baysysadmin

ASKER

I think you missunderstood me. I dont have apache in front on PROD.
Stage has apache and its working great. That server does exactly what you described above.

Im just going to nuke the server and start over.

This question needs an answer!

Become an EE member today

7 DAY FREE TRIAL

Members can start a 7-Day Free trial then enjoy unlimited access to the platform.

View membership options

Learn why we charge membership fees

We get it - no one likes a content blocker. Take one extra minute and find out why we block content.