asked on

Markview Application needs to have its process restarted every monday

Dear Experts,

I am troubleshooting an application that works with Oracle. It is called Markview 170 systems.

In the Redhat linux server, I have to restart the process that I see listed in crontabs we'll call it xdev. This happens every Monday.

This is done so the user can open images scan in the application

Where do I begin troubleshooting?

So far I get this:

Server 4 that has to be restarted

ps -ef | grep -i $USER | grep -i xdev

mvxpress 11664 1 0 12:43 pts/1 00:00:07 java -Xincgc -Xms1g -Xmx1g -Xss1m -Djava.awt.headless=true -DMvDataSourceLocation=jdbc/OracleDS -XX:NewSize=256m -XX:MaxNewSize=512m -Dmv.log.dir=log -jar oc4j.jar -config config-mvas-vue/server.xml -out log/mvas-vue.xdev.out -err log/mvas-vue.err
mvxpress 13657 18305 0 13:17 pts/0 00:00:00 grep -i xdev

This is from server 3 that does not need to be restarted:

ps -ef | grep -i $USER | grep -i xdev

ps -ef | grep -i $USER | grep -i xdev
mvasdocs 23595 1 0 12:43 pts/1 00:00:02 java -Djava.awt.headless=true -jar oc4j.jar -config config-mvas-vm/server.xml -out log/mvas-vm.xdev.out -err log/mvas-vm.err
mvasdocs 23600 1 0 12:43 pts/1 00:00:02 java -Djava.awt.headless=true -jar oc4j.jar -config config-graphics/server.xml -out log/graphics.xdev.out -err log/graphics.err
mvasdocs 23605 1 0 12:43 pts/1 00:00:02 java -Djava.awt.headless=true -jar oc4j.jar -config config-mvas-bcg/server.xml -out log/mvas-bcg.xdev.out -err log/mvas-bcg.err
mvasdocs 23618 1 0 12:43 pts/1 00:00:04 java -Djava.awt.headless=true -jar oc4j.jar -config config-mvas-dtm/server.xml -out log/mvas-dtm.xdev.out -err log/mvas-dtm.err
mvasdocs 23639 1 0 12:43 pts/1 00:00:04 java -Djava.awt.headless=true -jar oc4j.jar -config config-sfas-mm/server.xml -out ./log/sfas-mm.xdev.out -err ./log/sfas-mm.err

For any tips or solution on how to take on this I thank you!

M

Gerwin Jansen

Your server 4 has less processes running than server 3:

- is server 4 normally running 4 processes?
- can you have a look at the log folder, it contains several .err files, look at the end of those files, anything worth mentioning?
- how do you restart the processes? which commands do you have to give?

What 'client' applications do your users use? What kind (if any) error messages do they get? Can you post them>

Is server 4 experiencing high CPU load that you know of during certain periods? Do you have a system manager available that can assist you with this?

marceloNYC

ASKER

Nice! I will collect this information and get back to you.

marceloNYC

ASKER

Server 4 that has to be restart to fix the problem runs the express version not the complete version of the software.

I am attaching the script that either start or stops the ps.

That scripts calls for (not too sure) this process:

bbExecuteScript $Parm1-mvas-vm.sh >> $bbOc4jDbug
bbExecuteScript $Parm1-mvas-vm2.sh >> $bbOc4jDbug
bbExecuteScript $Parm1-mvas-vm3.sh >> $bbOc4jDbug
bbExecuteScript $Parm1-mvas-vue.sh >> $bbOc4jDbug
bbExecuteScript $Parm1-graphics.sh >> $bbOc4jDbug
bbExecuteScript $Parm1-mvas-bcg.sh >> $bbOc4jDbug
bbExecuteScript $Parm1-mvas-dtm.sh >> $bbOc4jDbug
# bbExecuteScript $Parm1-mvas-vm-monitor.sh >> $bbOc4jDbug
bbExecuteScript $Parm1-sfas-mm.sh >> $bbOc4jDbug

This is one log error with information for today:

4/07/21 12:08:12 OrionCMTConnection not closed, check your code!
14/07/21 12:08:12 Logical connection not closed, check your code!
14/07/21 12:08:12 (Use -Djdbc.connection.debug=true to find out where the leaked connection was created)
14/07/21 12:43:53 OrionCMTConnection not closed, check your code!
14/07/21 12:43:53 Logical connection not closed, check your code!
14/07/21 12:43:53 (Use -Djdbc.connection.debug=true to find out where the leaked connection was created)

from another .err log:

14/07/21 12:41:25 Warning: Error reading transaction-log file (/markview/mvxpress/oc4j_xdev/j2ee/home/persistence/transaction-mvas-vue.state) for recovery: premature end of file
14/07/21 12:41:25 Forced or abrupt (crash etc) server shutdown detected, starting recovery process...
14/07/21 12:41:25 Recovery completed, 0 connections committed and 0 rolled back...
14/07/21 12:41:35 *** (CRITICAL) Mon Jul 21 12:41:35 CDT 2014 [Thread[Orion Launcher,5,main]] java.lang.InstantiationException: jms.xml: port 9465 already in use.: java.net.BindException: Address already in use
      at java.net.PlainSocketImpl.socketBind(Native Method)
      at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:331)
      at java.net.ServerSocket.bind(ServerSocket.java:318)
      at java.net.ServerSocket.<init>(ServerSocket.java:185)
      at com.evermind.server.Server.createServerSocket(Server.java:165)
      at com.evermind.server.jms.JMSServer.initializeSocket(JMSServer.java:1143)
      at com.evermind.server.jms.JMSServer.setConfig(JMSServer.java:180)
      at com.evermind.server.ApplicationServer.initializeJMS(ApplicationServer.java:2363)
      at com.evermind.server.ApplicationServer.setConfig(ApplicationServer.java:1458)
      at com.evermind.server.ApplicationServerLauncher.run(ApplicationServerLauncher.java:92)
      at java.lang.Thread.run(Thread.java:534)

14/07/21 12:41:35 Error starting JMS-Server: jms.xml: port 9465 already in use.: java.net.BindException: Address already in use
      at java.net.PlainSocketImpl.socketBind(Native Method)
      at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:331)
      at java.net.ServerSocket.bind(ServerSocket.java:318)
      at java.net.ServerSocket.<init>(ServerSocket.java:185)
      at com.evermind.server.Server.createServerSocket(Server.java:165)
      at com.evermind.server.jms.JMSServer.initializeSocket(JMSServer.java:1143)
      at com.evermind.server.jms.JMSServer.setConfig(JMSServer.java:180)
      at com.evermind.server.ApplicationServer.initializeJMS(ApplicationServer.java:2363)
      at com.evermind.server.ApplicationServer.setConfig(ApplicationServer.java:1458)
      at com.evermind.server.ApplicationServerLauncher.run(ApplicationServerLauncher.java:92)
      at java.lang.Thread.run(Thread.java:534)

14/07/21 12:41:35 Error starting ORMI-Server. Unable to bind socket: Address already in use
14/07/21 12:42:09 Could not find mediaLib accelerator wrapper classes. Continuing in pure Java mode.
14/07/21 12:42:48 Warning: Error deleting the transaction-log file
14/07/21 12:44:33 Could not find mediaLib accelerator wrapper classes. Continuing in pure Java mode.
bbStartStop.sh

ASKER CERTIFIED SOLUTION

Gerwin Jansen

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

marceloNYC

ASKER

This is from yesterday's log information for bbStartStop.log

*************************************************************
*** 07/21/14 12:43:16 PM start
*** /markview/mvxpress/oc4j_xdev/j2ee/home
*************************************************************
bbWARNING: 'start-mvas-vm.sh' is not found OR is not hosted on this server.
bbWARNING: 'start-mvas-vm2.sh' is not found OR is not hosted on this server.
bbWARNING: 'start-mvas-vm3.sh' is not found OR is not hosted on this server.
170 MarkView Application Server Viewer Module started at Mon Jul 21 12:43:16 CDT 2014
bbWARNING: 'start-graphics.sh' is not found OR is not hosted on this server.
bbWARNING: 'start-mvas-bcg.sh' is not found OR is not hosted on this server.
bbWARNING: 'start-mvas-dtm.sh' is not found OR is not hosted on this server.
bbWARNING: 'start-sfas-mm.sh' is not found OR is not hosted on this server.
mvxpress 11664 1 39 12:43 pts/1 00:00:01 java -Xincgc -Xms1g -Xmx1g -Xss1m -Djava.awt.headless=true -DMvDataSourceLocation=jdbc/OracleDS -XX:NewSize=256m -XX:MaxNewSize=512m -Dmv.log.dir=log -jar oc4j.jar -config config-mvas-vue/server.xml -out log/mvas-vue.xdev.out -err log/mvas-vue.err

Gerwin Jansen

Looks like only one of the servers/processes is supposed to run on server 4.

Can you copy the existing bbStartStop.sh script to bbStartStop.sh.backup using:

cp -p bbStartStop.sh bbStartStop.sh.backup

and then edit bbStartStop.sh to exclude all but the one that is supposed to run, like this:

# bbExecuteScript $Parm1-mvas-vm.sh                                     >> $bbOc4jDbug
# bbExecuteScript $Parm1-mvas-vm2.sh                                    >> $bbOc4jDbug
# bbExecuteScript $Parm1-mvas-vm3.sh                                    >> $bbOc4jDbug
bbExecuteScript $Parm1-mvas-vue.sh                                    >> $bbOc4jDbug
# bbExecuteScript $Parm1-graphics.sh                                    >> $bbOc4jDbug
# bbExecuteScript $Parm1-mvas-bcg.sh                                    >> $bbOc4jDbug
# bbExecuteScript $Parm1-mvas-dtm.sh                                    >> $bbOc4jDbug
# bbExecuteScript $Parm1-mvas-vm-monitor.sh                             >> $bbOc4jDbug
# bbExecuteScript $Parm1-sfas-mm.sh                                     >> $bbOc4jDbug

Open in new window

Then stop / start again, look at the $bbOc4jDbug log and see if things are OK now?

marceloNYC

ASKER

Okay, I will need a little time.

marceloNYC

ASKER

Hello Gerwin Jansen,

Okay, here is an update. I think that we are almost there.

We found a different port that should be free. We think that port 9465 is used by another application and was enter in MarkView as the wrong port when updated back in January.

So we are going to reinstall the app with a proper free port. We think that fixes the issue.

Do you want closed the question now or rather wait to see how it goes?

Thank you for your help.

Gerwin Jansen

Thank you for your help.
Thank you :D

>> Do you want closed the question now or rather wait to see how it goes?
Well, that' up to you to be honest ;)

We'd like to know you've got it fixed. sure. But since there is a port conflict and that's what you're going to fix, the solution in itself is clear enough. If reinstalling takes a few days, just leave this question open, no problem.

marceloNYC

ASKER

Your help was great because I am new in the job. So when I pointed to my boss our the product of our discussion, he new the port is the issue, he has the experience with this installation. I will come back to this post to confirm or let you know if anything else happens.

Regards!

Gerwin Jansen

Glad we could help you out, thanks!

marceloNYC

ASKER

By the way, I think that you are right on the issue with the script.