troubleshooting Question

Domino Traveler quitting with Panic state

Avatar of captain
captainFlag for United Kingdom of Great Britain and Northern Ireland asked on
Email SoftwareLotus IBM
9 Comments1 Solution2515 ViewsLast Modified:
Hi

Problem:
We have a Domino mailserver and seen this error happening:
[1578:004B-17E4]  Thread=[1578:004B-17E4]
[1578:004B-17E4] Stack base=0x1CDE0088, Stack size = 7152 bytes
[1578:004B-17E4] PANIC: Process C:\Lotus\Domino\nTRAVELER.EXE (2444/0x98C) has terminated abnormally

The server is 8.5.3 FP5
Traveler is 8.5.3.3
running on Server 2003 SP2

Background:
We have upgraded both server and Traveler recently. Server from 8.5.3 and Traveler from 8.5.3.0. Both upgrades had to be done via Remote desktop. The upgrades were done with local admin account and both completed without error and server was rebooted after upgrade.

Fault searching:
We can reproduce the problem, as it appears to happen if a Remote Desktop session has ended. If admin logs on and stays logged on, server runs for hours without issue. The moment the admin logs off from remote session, the error occurs. Server needs brute force ending but sometimes NSD closes it.

This only happens when logging off NOT when disconnecting and keeping the session alive.

Attempted remedies:
We have unloaded Traveler and HTTP from notes.ini and defragged the Traveler db in line with this Technote as well as following through with deleting the NTSDB. We then reinstalled using local account and restarted the server.

The problem persisted. We then noticed that Domlog.nsf was at 5.6GB and deleted that for a rebuild on next restart. The problem seems to still exist.

I have just had to log in to get the logs and it reproduced the error. Shortly before the crash HTTP JVM reports:
HTTP JVM:  Exception Thrown: java.net.SocketException: Connection reset
[0FBC:003B-0A40] 30/08/2013 16:23:14   HTTP JVM: java.net.SocketException: Connection reset
[0FBC:003B-0A40] 30/08/2013 16:23:14   HTTP JVM:       at java.net.SocketInputStream.read(SocketInputStream.java:179)
[0FBC:003B-0A40] 30/08/2013 16:23:14   HTTP JVM:       at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2306)
[0FBC:003B-0A40] 30/08/2013 16:23:14   HTTP JVM:       at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2319)
[0FBC:003B-0A40] 30/08/2013 16:23:14   HTTP JVM:       at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2790)
[0FBC:003B-0A40] 30/08/2013 16:23:14   HTTP JVM:       at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:793)
[0FBC:003B-0A40] 30/08/2013 16:23:14   HTTP JVM:       at java.io.ObjectInputStream.<init>(ObjectInputStream.java:293)
[0FBC:003B-0A40] 30/08/2013 16:23:14   HTTP JVM:       at com.lotus.sync.servlet.ConnectionListener$CLWorker.run(ConnectionListener.java:1650)
[0FBC:003A-172C] 30/08/2013 16:23:14   HTTP JVM:  Exception Thrown: java.net.SocketException: Connection reset
[0FBC:003A-172C] 30/08/2013 16:23:14   HTTP JVM: java.net.SocketException: Connection reset
[0FBC:003A-172C] 30/08/2013 16:23:14   HTTP JVM:       at java.net.SocketInputStream.read(SocketInputStream.java:179)
[0FBC:003A-172C] 30/08/2013 16:23:14   HTTP JVM:       at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2306)
[0FBC:003A-172C] 30/08/2013 16:23:14   HTTP JVM:       at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2319)
[0FBC:003A-172C] 30/08/2013 16:23:14   HTTP JVM:       at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2790)
[0FBC:003A-172C] 30/08/2013 16:23:14   HTTP JVM:       at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:793)
[0FBC:003A-172C] 30/08/2013 16:23:14   HTTP JVM:       at java.io.ObjectInputStream.<init>(ObjectInputStream.java:293)
[0FBC:003A-172C] 30/08/2013 16:23:14   HTTP JVM:       at com.lotus.sync.servlet.ConnectionListener$CLWorker.run(ConnectionListener.java:1650)
[0FBC:0013-1338] 30/08/2013 16:23:20   HTTP JVM: Traveler: Lotus Traveler task did not respond.  Verify that the task is started.  Client request denied for CN=...

Additional info
We also see the following in the console, although it does not appear to have any effect on anything:
java.io.FileNotFoundException: D:\Domino\Data\traveler\cfg\NTSLogging.properties (The system cannot find the file specified.)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:137)
        at java.io.FileReader.<init>(FileReader.java:66)
        at com.lotus.sync.logging.XLogManager.updateLoggerConfig(XLogManager.java:753)
        at com.lotus.sync.logging.XLogManager.setActivityLevel(XLogManager.java:1593)
        at com.lotus.sync.admin.NTSSettingsINI.handleLoggerUpdates(NTSSettingsINI.java:1336)
        at com.lotus.sync.admin.NTSSettingsINI.refreshSettings(NTSSettingsINI.java:1239)
        at com.lotus.sync.admin.NTSSettingsINI.alarmCallback(NTSSettingsINI.java:1720)
        at com.lotus.sync.TSS.Util.WallClock$AlarmThreadRunnable.run(WallClock.java:103)
        at com.lotus.sync.util.ThreadPoolThread.run(ThreadPoolThread.java:391)

My question:
Has anyone got any ideas how to further diagnose?

Many thanks
capt.
Join the community to see this answer!
Join our exclusive community to see this answer & millions of others.
Unlock 1 Answer and 9 Comments.
Join the Community
Learn from the best

Network and collaborate with thousands of CTOs, CISOs, and IT Pros rooting for you and your success.

Andrew Hancock - VMware vExpert
See if this solution works for you by signing up for a 7 day free trial.
Unlock 1 Answer and 9 Comments.
Try for 7 days

”The time we save is the biggest benefit of E-E to our team. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange.

-Mike Kapnisakis, Warner Bros