troubleshooting Question

WebSphere - When Does websphere uses java trust store (cacerts) and its default Celldefaulttruststore

Avatar of Ravi Indukuri
Ravi IndukuriFlag for Malaysia asked on
Java
3 Comments1 Solution3232 ViewsLast Modified:
I have a weird problem in my websphere ND environments. My "A" websphere application needs to connect to "B" websphere application. I imported the B's Cert into "A" Truststore, But when env A tries to access env B. It is failing with the following exception.

But the weird part is not all the requests connecting to env B are failing with SSL exception, only few are failing. after troubleshooting i imported the root CA into jssecacerts of IBM Websphere and after restart the issue is solved.

I need your help to know when the IBM Websphere uses CACERTS, JSSECERTS and IBM TRUSTSTORE?

ERROR|2016-10-08 09:28:02,138|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 7|00099083|||com.mbb.wbs.dp3.decisionservice.MBBSORulesDP3ExecutorFailed to init executor
com.mbb.wbs.MBBSORulesExecException: javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.j: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
      java.security.cert.CertPathValidatorException: The certificate issued by CN=XXXXXXXX  is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.mbb.wbs.dp3.decisionservice.MBBSORulesDP3Executor.executeRules(MBBSORulesDP3Executor.java:82)
      at com.mbb.cmn.ConcreteMBBSORuleAppServerService.executeSORule(ConcreteMBBSORuleAppServerService.java:57)
      at com.mbb.cmn.MBBSORuleAppServerServiceBean.executeSORule(MBBSORuleAppServerServiceBean.java:50)
      at com.mbb.cmn.EJSRemoteStatelessMBBSORuleAppServerService_85df2dfe.executeSORule(Unknown Source)
      at com.mbb.cmn._MBBSORuleAppServerServiceRemote_Stub.executeSORule(_MBBSORuleAppServerServiceRemote_Stub.java:75)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:600)
      at com.s1.fst.httptunnel.delegate.DelegateEjbServiceBean$1.run(DelegateEjbServiceBean.java:228)
      at java.security.AccessController.doPrivileged(AccessController.java:310)
      at javax.security.auth.Subject.doAs(Subject.java:573)
      at com.ibm.websphere.security.auth.WSSubject.doAs(WSSubject.java:194)
      at com.ibm.websphere.security.auth.WSSubject.doAs(WSSubject.java:151)
      at com.s1.et.security.auth.spi.websphere.WSSubjectStrategy.doAs(WSSubjectStrategy.java:32)
      at com.s1.fst.httptunnel.delegate.DelegateEjbServiceBean.invoke(DelegateEjbServiceBean.java:236)
      at com.s1.fst.httptunnel.delegate.DelegateEjbServiceBean.invoke(DelegateEjbServiceBean.java:124)
      at com.s1.fst.httptunnel.delegate.EJSLocalStatelessDelegateEjbService_7ff94bbe.invoke(Unknown Source)
      at com.s1.fst.httptunnel.servlet.EjbInvokeHandler.process(EjbInvokeHandler.java:19)
      at com.s1.fst.httptunnel.servlet.HttpTunnelServlet.doPost(HttpTunnelServlet.java:137)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:738)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
      at com.ibm.ws.cache.servlet.ServletWrapper.serviceProxied(ServletWrapper.java:307)
      at com.ibm.ws.cache.servlet.CacheHook.handleFragment(CacheHook.java:576)
      at com.ibm.ws.cache.servlet.CacheHook.handleServlet(CacheHook.java:250)
      at com.ibm.ws.cache.servlet.ServletWrapper.service(ServletWrapper.java:259)
      at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1661)
      at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1602)
      at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:113)
      at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:80)
      at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:908)
      at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:939)
      at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:507)
      at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:181)
      at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:91)
      at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:878)
      at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1592)
      at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:191)
      at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:453)
      at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:515)
      at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:306)
      at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:84)
      at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1784)
      at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
      at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
      at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
      at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
      at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
      at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
      at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
      at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1656)
Caused by:
javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.j: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
      java.security.cert.CertPathValidatorException: The certificate issued by CN=******* Group Internal CA V1 is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)
      at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:154)
      at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
      at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
      at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
      at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
      at org.apache.axis.client.Call.invokeEngine(Call.java:2784)
      at org.apache.axis.client.Call.invoke(Call.java:2767)
      at org.apache.axis.client.Call.invoke(Call.java:1910)
      at com.mbb.wbs.dp3.decisionservice.DecisionServiceDP3SOAPProxy.executeDecisionService(DecisionServiceDP3SOAPProxy.java:66)
      at com.mbb.wbs.dp3.decisionservice.MBBSORulesDP3Executor.executeRules(MBBSORulesDP3Executor.java:69)
      ... 50 more
Caused by:
javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.j: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
      java.security.cert.CertPathValidatorException: The certificate issued by CN=XXXXXXX is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.ibm.jsse2.o.a(o.java:19)
      at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:689)
      at com.ibm.jsse2.kb.a(kb.java:271)
      at com.ibm.jsse2.kb.a(kb.java:516)
      at com.ibm.jsse2.lb.a(lb.java:59)
      at com.ibm.jsse2.lb.a(lb.java:274)
      at com.ibm.jsse2.kb.s(kb.java:167)
      at com.ibm.jsse2.kb.a(kb.java:484)
      at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:686)
      at com.ibm.jsse2.SSLSocketImpl.h(SSLSocketImpl.java:704)
      at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:12)
      at com.ibm.jsse2.SSLSocketImpl.startHandshake(SSLSocketImpl.java:498)
      at org.apache.axis.components.net.JSSESocketFactory.create(JSSESocketFactory.java:186)
      at org.apache.axis.transport.http.HTTPSender.getSocket(HTTPSender.java:191)
      at org.apache.axis.transport.http.HTTPSender.writeToSocket(HTTPSender.java:404)
      at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:138)
      ... 59 more
Caused by:
com.ibm.jsse2.util.j: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
      java.security.cert.CertPathValidatorException: The certificate issued by CN=****** Group Internal CA V1 is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.ibm.jsse2.util.h.b(h.java:104)
      at com.ibm.jsse2.util.h.b(h.java:14)
      at com.ibm.jsse2.util.g.a(g.java:3)
      at com.ibm.jsse2.pc.a(pc.java:11)
      at com.ibm.jsse2.pc.checkServerTrusted(pc.java:18)
      at com.ibm.jsse2.pc.b(pc.java:56)
      at com.ibm.jsse2.lb.a(lb.java:602)
      ... 70 more
Caused by:
java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
      java.security.cert.CertPathValidatorException: The certificate issued by CN=******* Group Internal CA V1 is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.ibm.security.cert.PKIXCertPathBuilderImpl.engineBuild(PKIXCertPathBuilderImpl.java:411)
      at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:258)
      at com.ibm.jsse2.util.h.b(h.java:108)
      ... 76 more
Caused by:
java.security.cert.CertPathValidatorException: The certificate issued by CN=****** Group Internal CA V1 is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.ibm.security.cert.BasicChecker.<init>(BasicChecker.java:111)
      at com.ibm.security.cert.PKIXCertPathValidatorImpl.engineValidate(PKIXCertPathValidatorImpl.java:178)
      at com.ibm.security.cert.PKIXCertPathBuilderImpl.myValidator(PKIXCertPathBuilderImpl.java:737)
      at com.ibm.security.cert.PKIXCertPathBuilderImpl.buildCertPath(PKIXCertPathBuilderImpl.java:649)
      at com.ibm.security.cert.PKIXCertPathBuilderImpl.engineBuild(PKIXCertPathBuilderImpl.java:357)
      ... 78 more
Caused by:
java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.ibm.security.cert.CertPathUtil.findIssuer(CertPathUtil.java:298)
      at com.ibm.security.cert.BasicChecker.<init>(BasicChecker.java:108)
      ... 82 more
INFO|2016-10-08 09:28:02,217|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  Transaction=Generic value for class com.mbb.dp3.pin.MBBPin ObjectId = null
INFO|2016-10-08 09:28:02,217|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  DrawerType={}
INFO|2016-10-08 09:28:02,217|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  Transaction=null
INFO|2016-10-08 09:28:02,217|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  Transaction=null
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  metaTransaction=com.s1.fst.transaction.meta.object.ConcreteTransaction@26752675
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  workflowName=dp3.mbbmykadpostworkflow
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>reverseMode>>>>>null
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>reenterMode>>>>>null
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>validateProcessTransaction>>>>>>Online
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>validateProcessTransaction>>isOnline4ServerSideUseOnly>>>>>>true
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  postTransactionResponse: return processTransaction(transaction, drawerType, parameters, workflowName);
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  transaction=Generic value for class com.mbb.dp3.pin.MBBPin ObjectId = null
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  drawerType=null
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  parameters={}
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  workflowName=dp3.mbbmykadpostworkflow
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||CALLING setDataForDynaTraceTag....
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  localUserService=com.s1.fst.app.user.ServerLocalUserService@2cc22cc2
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  currentStatus=memo
INFO|2016-10-08 09:28:02,229|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  parameters.put()={transaction=Generic value for class com.mbb.dp3.pin.MBBPin ObjectId = 345740617}
INFO|2016-10-08 09:28:02,229|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  getTransactionManager()= Null
INFO|2016-10-08 09:28:02,242|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||UsingEhCache>>>>>>> Object found with id == 36557, Name == CLK-BR-SGC
INFO|2016-10-08 09:28:02,243|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||nullisHostOverride >> 
INFO|2016-10-08 09:28:02,243|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>>>>>> MBBJournalStoreJournalEntryStrategy>>>>>>>>>>com.s1.fst.domain.journal.JournalEntryValue ID = null
INFO|2016-10-08 09:28:02,243|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>>>>>> MBBJournalStoreJournalEntryStrategy>>>>>>>>>>Supervisor Override required for non-STP Pin Maintenenace
ASKER CERTIFIED SOLUTION
AdminRAM

Our community of experts have been thoroughly vetted for their expertise and industry experience.

Join our community to see this answer!
Unlock 1 Answer and 3 Comments.
Start Free Trial
Learn from the best

Network and collaborate with thousands of CTOs, CISOs, and IT Pros rooting for you and your success.

Andrew Hancock - VMware vExpert
See if this solution works for you by signing up for a 7 day free trial.
Unlock 1 Answer and 3 Comments.
Try for 7 days

”The time we save is the biggest benefit of E-E to our team. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange.

-Mike Kapnisakis, Warner Bros