Avatar of bmcdowell540
bmcdowell540
Flag for United States of America asked on

Host IPS installed on Server -functionality of Web Application impaired w/ IPS, FW & AB off

I have a proprietary Web appliction running on Win 2008 x64, Tomcat 7 and Active MQ (JMS) as it is also a web service.   The client application connects to the Server using SSL via standard port 443 and to active MQ via the standard ssl port 61617.
 
Functionality between  works before I install the HIPS 7 module (latest patch).  Then, I install the HIPS module (IPS, NIPS, AB, and FW disabled/off) it still functions...UNTIL I restart the Tomcat 7 service.  At this point is when the connection between the client and server will fail when the conncetion is intitated by the client.  It appears someone (either client or server) closes the connection during the handshake.  
 
HIPS is NOT installed on the client to eliminate that from being the issue as we test and narrow the problem down..
 
I suspect is is an issue with SSL, I but need to know if HIPS, even turned off, changes anything with how the Server can use SSL configured a specific way so I can get the developers to fix it as the certificates and keystore are installed and configured using their own utilities in their application rather than manually.
 
Since it works before HIPS is installed up until Tomcat is restarted and then works after HIPS is uninstalled (with a reboot), everything is pointing to HIPS.  If I knew what could be blocked or not allowed even with all items turned off in HIPS, I could try to go that direction as well.
 
I have followed most of what is in this document except the Microsoft Debug Diagnostic Tool.  This also includes using wireshark and nothing stands out.
https://mysupport.mcafee.com/eservice/TemplatePage.aspx?sURL=3

 
All HIPS logs are put into verbose debut, but during this time and there are no VIOLATIONS picked up and there are no BLOCKED PID's.  Actually the logs really don't detect anything.
 
So, I have also tried putting FW with an ANY ANY ANY rule to see if it would work to narrow down the issue and perhaps see something in the logs.
 
Uninstalleing the NDIS driver did not help either among other steps I have take, which will take too much time to write out.
 
With all relevant troubleshooting steps I even took extra steps turning services on and off and rebooting to make sure.
 
I even resorted to trying HIPS 8 with everything off and had the same exact results.
 
 
Appreciate any help.
Anti-Virus AppsSoftware FirewallsApache Web Server

Avatar of undefined
Last Comment
traoher

8/22/2022 - Mon
traoher

You may want to check for port conflict.  Some of the applications may conflict with the port you are trying to use.
bmcdowell540

ASKER
I thought of that and changed some port numbers on the application.  

Do you know of a good way of checking port conflicts?
ASKER CERTIFIED SOLUTION
traoher

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question
I started with Experts Exchange in 2004 and it's been a mainstay of my professional computing life since. It helped me launch a career as a programmer / Oracle data analyst
William Peck