mod_jk errno=111 with apache2, tomcat6, on ubuntu 10.04

I am experiencing a periodic mod_jk failure which leads to a 503 site error. I can netstat to Tomcat on port 8009 during these failures, so I suspect that there is an issue with the apache => tomcat threads. I have yet to find a culprit thread in a Tomcat stack trace dump.

OS: Ubuntu 10.04
Web Server: Apache2
App Server: Tomcat6

Error from mod_jk.log:
ajp_connect_to_endpoint::jk_ajp_common(c) (922): Failed opening socket to (127.0.0.1:8009) (errno=111)

workers.properties:
worker.list=worker1
worker.worker1.port=8009
worker.worker1.host=localhost
worker.worker1.type=ajp13

Server.xml:
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
frankduxAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

LinuxGuruLinux Server AdministratorCommented:
Have you checked if some other process is listening on port 8009?

netstat -plan | grep ":8009"

Open in new window


Also if there is a firewall try stopping the firewall and see the results.

Try telnet to the port from the server itself.

telnet localhost 8009

Open in new window


Hope this helps.
Duncan RoeSoftware DeveloperCommented:
As root, you can lsof -i to see more detail about listening ports. Note that telnet localhost 8009 only tells you if a process is listening on the local interface (shows as lo in ifconfig) and you want to know about eth0 or whatever. Frequently processes listen on all interfaces, but this isn't necessarily the case
ramazanyichCommented:
I think it could be an issue with number of connections between apache and tomcat.
In tomcat you can add acceptCount attribute for the <Connector>. The defintion of that attribute is "The maximum queue length for incoming connection requests when all possible request processing threads are in use. Any requests received when the queue is full will be refused. Default value is 100" (check http://tomcat.apache.org/tomcat-6.0-doc/config/http.html for details)

Check the number of connections in Apache defined by MaxClients in your httpd.conf, default is 256. (check http://httpd.apache.org/docs/2.0/mod/mpm_common.html#maxclients).

If number of MaxClients is bigger that acceptCount then you can have such issues.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
frankduxAuthor Commented:
Thank you for the verbose explanation. There was indeed a mismatch between the Apache thread connections and the Tomcat max connections.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Apache Web Server

From novice to tech pro — start learning today.