Solved

Can't get Tomcat Manager to start on remote server

Posted on 2010-08-25
24
2,279 Views
Last Modified: 2013-12-02
I have been handed a server running RedHat with Apache/Tomcat in a test environment.  I have no experience with Tomcat.  I have one developer on the box who needs Tomcat's manager and it has stopped running but I can't find any error messages and I can't figure out how to get the manager to start.  Since I can't be at this server locally, I assume using a browser and typing http://serverIPaddress(or domain name)/{login sequence which I don't remember from work}will get me the same results but I haven't been able to make it work.  I apologize, but writing this question from home means I don't remember the message I got, either.  To make this clear, let's just call the server TEST.  I just need an understanding of the syntax of the URL to start the Tomcat manager.  Or, perhaps I've just shown that I need to know much more than that--if so, I humbly seek your education.
0
Comment
Question by:cbecker001
  • 14
  • 9
24 Comments
 
LVL 4

Expert Comment

by:oldmanbim
Comment Utility
I also am at home and run Tomcat on Windows, but I get to manager via http://localhost:8080/.  The default index has a link to it.  You'll have to know the admin password, which I believe is in your conf/tomcat-users.xml file.
0
 
LVL 26

Expert Comment

by:arober11
Comment Utility
0
 

Author Comment

by:cbecker001
Comment Utility
This gives me a direction to follow when I get to work.  I'll post the results then.

Thank you.
0
 

Author Comment

by:cbecker001
Comment Utility
Okay, I'm trying to connect with the Tomcat manager on this server by typing in http://test:8080 but get the message, "Unable to connect" from Firefox.  I've verified that 8080 is the port used by Tomcat, Apache is running, Tomcat is running, and there are still no errors in the log files pertaining to this.
0
 

Author Comment

by:cbecker001
Comment Utility
Does this look correct for my tomcat-users.xml file? (besides the bad passwords)

<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
  <role rolename="tomcat"/>
  <role rolename="role1"/>
  <role rolename="manager"/>
  <role rolename="admin"/>
  <user username="tomcat" password="tomcat" roles="tomcat"/>
  <user username="role1" password="tomcat" roles="role1"/>
  <user username="both" password="tomcat" roles="tomcat,role1"/>
  <user username="manager" password="tomcat" roles="manager"/>
  <user username="admin" password="tomcat" roles="admin"/>
</tomcat-users>

Because still, when I put this in the URL:  http://test:8080/admin
All I get in the browser is the following:

Unable to connect
Firefox can't establish a connection to the server at test:8080.

However, I can ssh into the server--obviously I got the printout of the file above.  So connecting to the server in general is not a problem.
0
 
LVL 26

Accepted Solution

by:
arober11 earned 500 total points
Comment Utility
On the server can you please let us know what you get from entering the following as Root:

1)  First type:

netstat -plant | grep 8080


# Do you see a:

tcp        0      0 :::8080                 :::*                    LISTEN      1234/java

#Or a:
tcp        0      0 127.0.0.1:8080      :::*                    LISTEN      1234/java

#Or a:
tcp        0      0 11.22.33.44:8080   :::*                    LISTEN      1234/java


2) If you don't see any of the above, see if there are any java processes running, via a:

ps -ef | grep -i java


3) If there are and ones a Tomcat change into the "logs" directory of the catalina home, if your not sure where this is it should appear as one of the command line options in the "ps" output above e.g.   -Dcatalina.home=/usr/share/tomcat6

Then type:

egrep "ERROR|WARNING" catalina.out

0
 

Author Comment

by:cbecker001
Comment Utility
1) netstat -plant | grep 8080'

returned:

tcp        0      0 :::8080                     :::*                        LISTEN      10472/java


2) ps -ef|grep java

returned

tomcat   10472     1  0 13:24 ?        00:00:06 /usr/lib/jvm/java/bin/java -Dcatalina.ext.dirs=/usr/share/tomcat5/shared/lib:/usr/share/tomcat5/common/lib -Dcatalina.ext.dirs=/usr/share/tomcat5/shared/lib:/usr/share/tomcat5/common/lib -Djava.endorsed.dirs=/usr/share/tomcat5/common/endorsed -classpath /usr/lib/jvm/java/lib/tools.jar:/usr/share/tomcat5/bin/bootstrap.jar:/usr/share/tomcat5/bin/commons-logging-api.jar:/usr/share/java/mx4j/mx4j-impl.jar:/usr/share/java/mx4j/mx4j-jmx.jar -Dcatalina.base=/usr/share/tomcat5 -Dcatalina.home=/usr/share/tomcat5 -Djava.io.tmpdir=/usr/share/tomcat5/temp org.apache.catalina.startup.Bootstrap start


3) egrep "ERROR|WARNING" catalina.out returned nothing.  However, there were lines with "SEVERE" that I have pasted below.  These weren't there when I looked previously:


SEVERE: The scratchDir you specified: /usr/share/tomcat5/work/Catalina/localhost/admin is unusable.
SEVERE: The scratchDir you specified: /usr/share/tomcat5/work/Catalina/localhost/host-manager is unusable.
SEVERE: The scratchDir you specified: /usr/share/tomcat5/work/Catalina/localhost/balancer is unusable.
SEVERE: IOException while saving persisted sessions: java.io.FileNotFoundException: /usr/share/tomcat5/work/Catalina/localhost/admin/SESSIONS.ser (Permission denied)
SEVERE: Exception unloading sessions to persistent storage
SEVERE: IOException while saving persisted sessions: java.io.FileNotFoundException: /usr/share/tomcat5/work/Catalina/localhost/balancer/SESSIONS.ser (Permission denied)
SEVERE: Exception unloading sessions to persistent storage
SEVERE: IOException while saving persisted sessions: java.io.FileNotFoundException: /usr/share/tomcat5/work/Catalina/localhost/host-manager/SESSIONS.ser (Permission denied)
SEVERE: Exception unloading sessions to persistent storage
SEVERE: The scratchDir you specified: /usr/share/tomcat5/work/Catalina/localhost/admin is unusable.
SEVERE: The scratchDir you specified: /usr/share/tomcat5/work/Catalina/localhost/host-manager is unusable.
SEVERE: The scratchDir you specified: /usr/share/tomcat5/work/Catalina/localhost/balancer is unusable.
SEVERE: IOException while saving persisted sessions: java.io.FileNotFoundException: /usr/share/tomcat5/work/Catalina/localhost/admin/SESSIONS.ser (Permission denied)
SEVERE: Exception unloading sessions to persistent storage
SEVERE: IOException while saving persisted sessions: java.io.FileNotFoundException: /usr/share/tomcat5/work/Catalina/localhost/balancer/SESSIONS.ser (Permission denied)
SEVERE: Exception unloading sessions to persistent storage
SEVERE: IOException while saving persisted sessions: java.io.FileNotFoundException: /usr/share/tomcat5/work/Catalina/localhost/host-manager/SESSIONS.ser (Permission denied)
SEVERE: Exception unloading sessions to persistent storage
SEVERE: The scratchDir you specified: /usr/share/tomcat5/work/Catalina/localhost/admin is unusable.
SEVERE: The scratchDir you specified: /usr/share/tomcat5/work/Catalina/localhost/host-manager is unusable.
SEVERE: The scratchDir you specified: /usr/share/tomcat5/work/Catalina/localhost/balancer is unusable.
0
 
LVL 26

Assisted Solution

by:arober11
arober11 earned 500 total points
Comment Utility
I'm guessing you've previously run Tomcat as "root" rather than "tomcat", the user-id it's currently running under, so one or more files or directories are only accessible to the "root" user id. If the following returns any output my assumption is correct:

find /usr/share/tomcat5/work/ ! -group tomcat -a ! -user tomcat  -a ! -type l -follow -ls


In which case you'll want to stop Tomcat and fix the permissions e.g.

/etc/init.d/tomcat* stop
chgrp tomcat  /usr/share/tomcat5/work/*
chmof g+w /usr/share/tomcat5/work/*
cd /usr/share/tomcat5/logs/
mv catalina.out catalina.out.last
/etc/init.d/tomcat* start
0
 
LVL 26

Assisted Solution

by:arober11
arober11 earned 500 total points
Comment Utility
Woops, missing a couple of -R (recursive), should have read:

/etc/init.d/tomcat* stop
chgrp -R tomcat  /usr/share/tomcat5/work/*
chmod  -R g+w /usr/share/tomcat5/work/*
cd /usr/share/tomcat5/logs/
mv catalina.out catalina.out.last
/etc/init.d/tomcat* start
0
 

Author Comment

by:cbecker001
Comment Utility
You are correct--I got output from that command.

I have to leave for the day, but I'll work on this tomorrow.  Thank you so much for your help so far.  
0
 

Author Comment

by:cbecker001
Comment Utility
Okay, I made all of the ownership and permission changes you suggested.  I restarted tomcat but I still can't get any response from http://test:8080.  Here is my tomcat-users.xml again.  I'm just not sure that this is correct:

<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
  <role rolename="tomcat"/>
  <role rolename="role1"/>
  <role rolename="manager"/>
  <role rolename="admin"/>
  <user username="tomcat" password="tomcat" roles="tomcat,manager"/>
  <user username="role1" password="tomcat" roles="role1"/>
  <user username="both" password="tomcat" roles="tomcat,role1"/>
  <user username="manager" password="tomcat" roles="manager"/>
</tomcat-users>


Also, would these permissions in my 'conf' directory be causing any problems:

/usr/share/tomcat5/conf]# ls -al
drwxrwxr-x  3 root   tomcat  4096 Aug 27 08:19 .
drwxr-xr-x 98 root   root   12288 Aug 27 07:50 ..
drwxrwxr-x  3 root   tomcat  4096 Jul 29 10:31 Catalina
-rw-r--r--  1 root   tomcat  8635 Jul 29 10:31 catalina.policy
-rw-r--r--  1 root   tomcat  3114 Jul 29 10:31 catalina.properties
-rw-r--r--  1 root   root     330 Jul 29 10:31 context.xml
-rw-r--r--  1 root   root     310 Jul 29 10:30 jk2.manifest
-rw-rw----  1 root   tomcat   778 Jul 29 10:30 jk2.properties
-rw-r--r--  1 root   root    1981 Jul 29 10:30 jkconf.ant.xml
-rw-r--r--  1 root   root     170 Jul 29 10:30 jkconfig.manifest
-rw-rw----  1 root   tomcat  2824 Jul 29 10:31 logging.properties
-rw-r--r--  1 root   root     851 Jul 29 10:31 server-minimal.xml
-rw-r--r--  1 root   root   18854 Dec 16  2009 server.xml
-rw-r--r--  1 root   root     125 Jul 29 10:30 shm.manifest
-rw-r--r--  1 root   root    1520 Dec 16  2009 tomcat5.conf
-rw-r--r--  1 root   root     243 Jul 29 10:30 tomcat-jk2.manifest
-rw-r--r--  1 tomcat tomcat   437 Aug 27 07:49 tomcat-users.xml
-rw-r--r--  1 root   root     563 Jul 29 10:30 uriworkermap.properties
-rw-r--r--  1 root   root   49386 Jul 29 10:31 web.xml
-rw-r--r--  1 root   root    2793 Jul 29 10:30 workers2.properties
-rw-r--r--  1 root   root     871 Jul 29 10:30 workers2.properties.minimal
-rw-r--r--  1 root   root    5667 Jul 29 10:30 workers.properties
-rw-r--r--  1 root   root     535 Jul 29 10:30 workers.properties.minimal
0
 
LVL 26

Expert Comment

by:arober11
Comment Utility
Your tomcat-users.xml still looks fine, and wouldn't stop tomcat from starting anyway. What's in the catalina.out when you attempt to re-start the server, per my example above its advisable to move the existing log out of the way after shutting the Tomcat down, and before re-starting.
0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 

Author Comment

by:cbecker001
Comment Utility
There are no errors in catalina.out but there are some INFO messages.  The file isn't that long so I'm posting it here:

Using CATALINA_BASE:   /usr/share/tomcat5
Using CATALINA_HOME:   /usr/share/tomcat5
Using CATALINA_TMPDIR: /usr/share/tomcat5/temp
Using JRE_HOME:
Aug 27, 2010 7:40:26 AM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/jdk1.5.0_14/jre/lib/i386/server:/usr/java/jdk1.5.0_14/jre/lib/i386:/usr/java/jdk1.5.0_14/jre/../lib/i386
Aug 27, 2010 7:40:26 AM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-8080
Aug 27, 2010 7:40:26 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 961 ms
Aug 27, 2010 7:40:26 AM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Aug 27, 2010 7:40:26 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.23
Aug 27, 2010 7:40:27 AM org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
Aug 27, 2010 7:40:29 AM org.apache.catalina.core.ApplicationContext log
INFO: org.apache.webapp.balancer.BalancerFilter: init(): ruleChain: [org.apache.webapp.balancer.RuleChain: [org.apache.webapp.balancer.rules.URLStringMatchRule: Target string: News / Redirect URL: http://www.cnn.com], [org.apache.webapp.balancer.rules.RequestParameterRule: Target param name: paramName / Target param value: paramValue / Redirect URL: http://www.yahoo.com], [org.apache.webapp.balancer.rules.AcceptEverythingRule: Redirect URL: http://jakarta.apache.org]]
Aug 27, 2010 7:40:29 AM org.apache.coyote.http11.Http11BaseProtocol start
INFO: Starting Coyote HTTP/1.1 on http-8080
Aug 27, 2010 7:40:29 AM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:8009
Aug 27, 2010 7:40:29 AM org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/16  config=null
Aug 27, 2010 7:40:29 AM org.apache.catalina.storeconfig.StoreLoader load
INFO: Find registry server-registry.xml at classpath resource
Aug 27, 2010 7:40:30 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 3192 ms
Using CATALINA_BASE:   /usr/share/tomcat5
Using CATALINA_HOME:   /usr/share/tomcat5
Using CATALINA_TMPDIR: /usr/share/tomcat5/temp
Using JRE_HOME:
Aug 27, 2010 7:49:51 AM org.apache.coyote.http11.Http11BaseProtocol pause
INFO: Pausing Coyote HTTP/1.1 on http-8080
Aug 27, 2010 7:49:52 AM org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
Aug 27, 2010 7:49:52 AM org.apache.coyote.http11.Http11BaseProtocol destroy
INFO: Stopping Coyote HTTP/1.1 on http-8080
Aug 27, 2010 7:49:52 AM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: Failed shutdown of Apache Portable Runtime
Using CATALINA_BASE:   /usr/share/tomcat5
Using CATALINA_HOME:   /usr/share/tomcat5
Using CATALINA_TMPDIR: /usr/share/tomcat5/temp
Using JRE_HOME:
Aug 27, 2010 7:49:57 AM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/jdk1.5.0_14/jre/lib/i386/server:/usr/java/jdk1.5.0_14/jre/lib/i386:/usr/java/jdk1.5.0_14/jre/../lib/i386
Aug 27, 2010 7:49:57 AM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-8080
Aug 27, 2010 7:49:57 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 676 ms
Aug 27, 2010 7:49:57 AM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Aug 27, 2010 7:49:57 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.23
Aug 27, 2010 7:49:57 AM org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
Aug 27, 2010 7:49:58 AM org.apache.catalina.core.ApplicationContext log
INFO: org.apache.webapp.balancer.BalancerFilter: init(): ruleChain: [org.apache.webapp.balancer.RuleChain: [org.apache.webapp.balancer.rules.URLStringMatchRule: Target string: News / Redirect URL: http://www.cnn.com], [org.apache.webapp.balancer.rules.RequestParameterRule: Target param name: paramName / Target param value: paramValue / Redirect URL: http://www.yahoo.com], [org.apache.webapp.balancer.rules.AcceptEverythingRule: Redirect URL: http://jakarta.apache.org]]
Aug 27, 2010 7:49:59 AM org.apache.coyote.http11.Http11BaseProtocol start
INFO: Starting Coyote HTTP/1.1 on http-8080
Aug 27, 2010 7:49:59 AM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:8009
Aug 27, 2010 7:49:59 AM org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/90  config=null
Aug 27, 2010 7:49:59 AM org.apache.catalina.storeconfig.StoreLoader load
INFO: Find registry server-registry.xml at classpath resource
Aug 27, 2010 7:49:59 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 1911 ms
0
 
LVL 26

Expert Comment

by:arober11
Comment Utility
Goot the errors are gone, next do you have a:

conf/Catalina/localhost/manager.xml

If so what's in the file?

Can you access: http://localhost:8080/manager

See: http://tomcat.apache.org/tomcat-5.5-doc/manager-howto.html
0
 

Author Comment

by:cbecker001
Comment Utility
Contents of "/etc/tomcat5/Catalina/localhost"  (notice the different path)
(some blank lines removed for brevity)

<!--
    Context configuration file for the Tomcat Manager Web App
    $Id: manager.xml 303123 2004-08-26 17:03:35Z remm $
-->
<Context docBase="${catalina.home}/server/webapps/manager"
         privileged="true" antiResourceLocking="false" antiJARLocking="false">

  <!-- Link to the user database we will get roles from -->
  <ResourceLink name="users" global="UserDatabase"
                type="org.apache.catalina.UserDatabase"/>

</Context>


If I try to access "http://localhost:8080/manager" I still get the same error in Firefox.

I have to leave work early today so I'll read up on your link and work on it over the weekend.

Thanks again for your help.
0
 

Author Comment

by:cbecker001
Comment Utility
Your link to http://tomcat.apache.org/tomcat-5.5-doc/manager-howto.html is helpful, but it has me more confused than ever.  It states that the manager.xml file should have this in it:

   <Context path="/manager" debug="0" privileged="true"
            docBase="/usr/local/kinetic/tomcat5/server/webapps/manager">
   </Context>


However, the manager.xml file on my server looks like the following:


<Context docBase="${catalina.home}/server/webapps/manager"
         privileged="true" antiResourceLocking="false" antiJARLocking="false">

  <!-- Link to the user database we will get roles from -->
  <ResourceLink name="users" global="UserDatabase"
                type="org.apache.catalina.UserDatabase"/>

</Context>


How did ours ever work in the past without things like the path being set?
0
 
LVL 26

Assisted Solution

by:arober11
arober11 earned 500 total points
Comment Utility
Hi, the path (docBase) is optional, if it isn't specified Tomcat will look in the Tomcat $CATALINA_HOME/webapps/xxxxxx for the application, where xxxxxx is the app name.

So you could get away with a manager.xml file of:


<?xml version="1.0" encoding="UTF-8"?>
<Context antiResourceLocking="false" privileged="true">
</Context>

Try for yourself, back-up the existing file, replace with the single entry above, and bounce the tomcat.
0
 

Author Comment

by:cbecker001
Comment Utility
I tried copying in your three lines and restarting tomcat but couldn't get anything with "http://test:8080/manager" or "http://test:8080/".

Another thing that I'm wondering about this is how this could have stopped working so abruptly and now I find so many things wrong.  I would expect to find only a single thing incorrect--or maybe two.  But it seems that there have been significant problems--which again begs the question of, "How could it have worked before?
0
 
LVL 26

Assisted Solution

by:arober11
arober11 earned 500 total points
Comment Utility
So far only one issue, file permissions, have been identified, which was probably down to the Tomcat being manually run from root. On which subject the following may be worth a try:

/etc/init.d/tomcat* stop
cd  /usr/share/tomcat5
mv work work.old
mkdir work
chown tomcat work
/etc/init.d/tomcat* start
sleep 20
telnet localhost 8080
GET /
0
 

Author Comment

by:cbecker001
Comment Utility
# telnet localhost 8080
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
GET /
Connection closed by foreign host.

I don't know why I didn't think of this earlier, but I compared all of the files under /etc/tomcat5 between test and production and they are all identical.

0
 

Author Comment

by:cbecker001
Comment Utility
I can't help but think that this problem must be much simpler than this.  I did ask the two people who use this box what they were doing right before this happened but neither remembers anything out of the ordinary.  This was running for years and then suddenly stopped--no administrative activity on the box at all.  The problem must be more obvious than this.

Hmmmmmm.......
0
 

Author Closing Comment

by:cbecker001
Comment Utility
Extremely knowledgeable.  Delved into details that helped stabilize my software for the future--not just for this problem
0
 
LVL 26

Expert Comment

by:arober11
Comment Utility
Hi, from the catalina / telnet output you appear to have no applications deployed, hence my last suggestion of blowing away the work directory to try and force the application to re-deploy, on re-start. If the work directory is empty after the re-start / you still have no manager app have a look at your conf/server.xml do you have an unpackWARs and autodeploy attribute foe the localhost defenition?

0
 

Author Comment

by:cbecker001
Comment Utility
arober11, I don't know how to answer that question by examining my server.xml file.  Can I email it to you?  I don't want to post it here for privacy reasons.  It is 380 lines long.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

This article is about some of the basic and important steps to be used to improve the performance in web-sphere commerce application development. 1) Always leverage the Dyna-caching facility provided by the product 2) Remove the unwanted code …
If your site has a few sections that need to be secure when data is transmitted between the server and local computer, such as a /order/ section for ordering or /customer/ which contains customer data, etc it would of course be recommended to secure…
Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now