Need to eliminate HTTP Status 500 errors...

I have a web based java application running on multiple application servers.  All experience the same issue(s).  The heart of the problem is that while using the application, at any given time (fairly intermittent, no discernible pattern that we see) trying to get to no particular page, nor execute any specific functionality, we can get greeted with a page indicating:

HTTP Status 500 - Trying to process transaction <whatever we were doing / page we were trying to get to>
type: Status Report
message: Trying to process <fill in the action here>
description: The server encountered an internal error that prevented it from fulfilling this request.

Apache Tomcat 5.0.28  

A few very important points of note (I think if we solve #2- we solve the issue):
       1. Seems to be less frequent, but still happens, when there is less load (fewer users....less network traffic....however you want to view it).
       2. The application has a splash/home page that contains a window/box that is used for putting out announcements to the users.  When using IE as the browser and accessing the server/page using the server name - this box/window within the page displays this same HTTP 500 error ALL THE TIME without fail.   When using IE and accessing the server/page by IP, the announcement page displays correctly in the window ALL THE TIME without fail.   When using Firefox to access the server/page, this announcement window/box ALWAYS works regardless of using server name or IP.  (but, yes, we still do get the intermittent 500's using Firefox).
     3.  The application developers have this deployed at several other locations and indication is that none of them have ever experienced this, mean the 500's in the app or in the announcement box (and I do trust those telling me this).  They have reviewed the application, etc. and they feel it is something in the environment (network, DNS, etc.)
   4. Somewhat contradicting it being load/traffic related is the fact that the application screens are designed to continually refresh themselves (and they do) - meaning, if I were to leave a page up throughout the night it should still be up in the morning - but what we experience is that at some point during the evening (and there is next to no load and no users during the night) the refresh will come back with the 500 error.
  5. When we get the 500- we can usually just refresh the page (re-posting data or resending when need be) and more often than not, everything will be fine and the page will load- although we have had it occur where it happens multiple times in a row (but pretty infrequently).

We've tried all sort of things with DNS, network tweaking, etc. - and I am sure I'm forgetting some points of note above - but nothing we have tried has rid us of the problem.

Any help that can be offered is desperately appreciated...

Thanks so much...

derdleAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

nasserdCommented:
It sounds like a server memory issue -- add more RAM and make sure the server app recycles its memory usage more frequently.
0
derdleAuthor Commented:
We have monitored memory and CPU very closely on the servers and see nothing to indicate an issue in these areas.

Also, that certainly would not be related in regards to the announcements box showing the error in IE when using the server name, but not when using the IP, and Firefox not showing the error at all using name or IP - that somewhat speaks against it being memory as well.  Don't you think...

Not to mention that the servers are 64bit and we already have 8GB in them.
0
derdleAuthor Commented:
OK- here's a better explanation of the Announcements window, and also wanted to provide further information (I said I knew I'd forget some things).

The HTTP 500 error as it appears in the Announcements window is "slightly" different than the one received using the application.  The difference being:

Occurs in the app:

HTTP Status 500 - Trying to process transaction <whatever we were doing / page we were trying to get to>
type: Status Report
message: Trying to process <fill in the action here>
description: The server encountered an internal error that prevented it from fulfilling this request.
Apache Tomcat 5.0.28

As it appears in the Announcements box when using IE and calling the server by name

HTTP Status 500 -
type: Status Report
message:
description: The server encountered an internal error that prevented it from fulfilling this request.
Apache Tomcat 5.0.28  

basically- no description in the header, and no message text in the Announcement frame

better description of the Announcement box -

Its basically an IFRAME within the page. The IFRAME is a way of loading content into a certain area of a page. THE IFRAME calls a process on the webserver to render its output into the frame..

<iframe src="/sourcedir/broadcastmsg"; style="height:71; width:99%" name="I1">
            [Your browser does < em>not</em>
            support <code>iframe</code>,
            or has been configured not to display inline frames.]</iframe>

where src="/sourcedir/broadcastmsg"; is the call to the process (a servlet) to render the output.

Thanks for any continuing thoughts...

 

0
derdleAuthor Commented:
Another significant detail....  The only difference the devs know between our environment and the other locations where this is deployed (that never have the issue) is the fact that our servers are running 64 bit.

So, the leading question becomes-

Does jdk1.5_17(32 bit JVM) run on Windows 64 bit OS without any known issues? Specifically class loading?  Anything were missing? Unique settings?

We found instances of this buried in the logs...class load failures


> java.net.URLClassLoader$1.run(URLClassLoader.java:200)

>  >>>>>>>


> INFO   | jvm 1    | 2009/02/02 10:29:53 | 10:29:53,140 ERROR [Engine]
> StandardContext[/ansHealthConsole]Error loading
> org.jboss.web.tomcat.tc5.WebCtxLoader$ENCLoader@5db132
> console.server.servlets.AnnouncementHandler
> INFO   | jvm 1    | 2009/02/02 10:29:53 |
> java.lang.ClassNotFoundException:
> console.server.servlets.AnnouncementHandler
> INFO   | jvm 1    | 2009/02/02 10:29:53 |     at
>  >>>java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> INFO   | jvm 1    | 2009/02/02 10:29:53 |     at
> java.security.AccessController.doPrivileged(Native Method)
> INFO   | jvm 1    | 2009/02/02 10:29:53 |     at
> java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> INFO   | jvm 1    | 2009/02/02 10:29:53 |     at
> java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> INFO   | jvm 1    | 2009/02/02 10:29:53 |     at
> java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> INFO   | jvm 1    | 2009/02/02 10:29:53 |     at
> org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:964)

> INFO   | jvm 1    | 2009/02/02 10:29:53 |     at
> org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:862)
> INFO   | jvm 1    | 2009/02/02 10:29:53 |     at
> org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4013)

> INFO   | jvm 1    | 2009/02/02 10:29:53 |     at
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4357)

Again, any help if appreciated.   I'll go post this question separately as well....
0
derdleAuthor Commented:
No other feedback has been provided...and this is still an open issue, but given the lack of any reponses- please close this item.  Not sure how to do so without accepting a solution....for which one was not provided.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java App Servers

From novice to tech pro — start learning today.