We help IT Professionals succeed at work.
Get Started

WebSphere - When Does websphere uses java trust store (cacerts) and its default Celldefaulttruststore

Ravi Indukuri
on
3,227 Views
Last Modified: 2017-04-09
I have a weird problem in my websphere ND environments. My "A" websphere application needs to connect to "B" websphere application. I imported the B's Cert into "A" Truststore, But when env A tries to access env B. It is failing with the following exception.

But the weird part is not all the requests connecting to env B are failing with SSL exception, only few are failing. after troubleshooting i imported the root CA into jssecacerts of IBM Websphere and after restart the issue is solved.

I need your help to know when the IBM Websphere uses CACERTS, JSSECERTS and IBM TRUSTSTORE?

ERROR|2016-10-08 09:28:02,138|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 7|00099083|||com.mbb.wbs.dp3.decisionservice.MBBSORulesDP3ExecutorFailed to init executor
com.mbb.wbs.MBBSORulesExecException: javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.j: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
      java.security.cert.CertPathValidatorException: The certificate issued by CN=XXXXXXXX  is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.mbb.wbs.dp3.decisionservice.MBBSORulesDP3Executor.executeRules(MBBSORulesDP3Executor.java:82)
      at com.mbb.cmn.ConcreteMBBSORuleAppServerService.executeSORule(ConcreteMBBSORuleAppServerService.java:57)
      at com.mbb.cmn.MBBSORuleAppServerServiceBean.executeSORule(MBBSORuleAppServerServiceBean.java:50)
      at com.mbb.cmn.EJSRemoteStatelessMBBSORuleAppServerService_85df2dfe.executeSORule(Unknown Source)
      at com.mbb.cmn._MBBSORuleAppServerServiceRemote_Stub.executeSORule(_MBBSORuleAppServerServiceRemote_Stub.java:75)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:600)
      at com.s1.fst.httptunnel.delegate.DelegateEjbServiceBean$1.run(DelegateEjbServiceBean.java:228)
      at java.security.AccessController.doPrivileged(AccessController.java:310)
      at javax.security.auth.Subject.doAs(Subject.java:573)
      at com.ibm.websphere.security.auth.WSSubject.doAs(WSSubject.java:194)
      at com.ibm.websphere.security.auth.WSSubject.doAs(WSSubject.java:151)
      at com.s1.et.security.auth.spi.websphere.WSSubjectStrategy.doAs(WSSubjectStrategy.java:32)
      at com.s1.fst.httptunnel.delegate.DelegateEjbServiceBean.invoke(DelegateEjbServiceBean.java:236)
      at com.s1.fst.httptunnel.delegate.DelegateEjbServiceBean.invoke(DelegateEjbServiceBean.java:124)
      at com.s1.fst.httptunnel.delegate.EJSLocalStatelessDelegateEjbService_7ff94bbe.invoke(Unknown Source)
      at com.s1.fst.httptunnel.servlet.EjbInvokeHandler.process(EjbInvokeHandler.java:19)
      at com.s1.fst.httptunnel.servlet.HttpTunnelServlet.doPost(HttpTunnelServlet.java:137)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:738)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
      at com.ibm.ws.cache.servlet.ServletWrapper.serviceProxied(ServletWrapper.java:307)
      at com.ibm.ws.cache.servlet.CacheHook.handleFragment(CacheHook.java:576)
      at com.ibm.ws.cache.servlet.CacheHook.handleServlet(CacheHook.java:250)
      at com.ibm.ws.cache.servlet.ServletWrapper.service(ServletWrapper.java:259)
      at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1661)
      at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1602)
      at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:113)
      at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:80)
      at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:908)
      at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:939)
      at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:507)
      at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:181)
      at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:91)
      at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:878)
      at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1592)
      at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:191)
      at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:453)
      at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:515)
      at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:306)
      at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:84)
      at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1784)
      at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
      at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
      at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
      at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
      at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
      at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
      at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
      at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1656)
Caused by:
javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.j: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
      java.security.cert.CertPathValidatorException: The certificate issued by CN=******* Group Internal CA V1 is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)
      at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:154)
      at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
      at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
      at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
      at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
      at org.apache.axis.client.Call.invokeEngine(Call.java:2784)
      at org.apache.axis.client.Call.invoke(Call.java:2767)
      at org.apache.axis.client.Call.invoke(Call.java:1910)
      at com.mbb.wbs.dp3.decisionservice.DecisionServiceDP3SOAPProxy.executeDecisionService(DecisionServiceDP3SOAPProxy.java:66)
      at com.mbb.wbs.dp3.decisionservice.MBBSORulesDP3Executor.executeRules(MBBSORulesDP3Executor.java:69)
      ... 50 more
Caused by:
javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.j: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
      java.security.cert.CertPathValidatorException: The certificate issued by CN=XXXXXXX is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.ibm.jsse2.o.a(o.java:19)
      at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:689)
      at com.ibm.jsse2.kb.a(kb.java:271)
      at com.ibm.jsse2.kb.a(kb.java:516)
      at com.ibm.jsse2.lb.a(lb.java:59)
      at com.ibm.jsse2.lb.a(lb.java:274)
      at com.ibm.jsse2.kb.s(kb.java:167)
      at com.ibm.jsse2.kb.a(kb.java:484)
      at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:686)
      at com.ibm.jsse2.SSLSocketImpl.h(SSLSocketImpl.java:704)
      at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:12)
      at com.ibm.jsse2.SSLSocketImpl.startHandshake(SSLSocketImpl.java:498)
      at org.apache.axis.components.net.JSSESocketFactory.create(JSSESocketFactory.java:186)
      at org.apache.axis.transport.http.HTTPSender.getSocket(HTTPSender.java:191)
      at org.apache.axis.transport.http.HTTPSender.writeToSocket(HTTPSender.java:404)
      at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:138)
      ... 59 more
Caused by:
com.ibm.jsse2.util.j: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
      java.security.cert.CertPathValidatorException: The certificate issued by CN=****** Group Internal CA V1 is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.ibm.jsse2.util.h.b(h.java:104)
      at com.ibm.jsse2.util.h.b(h.java:14)
      at com.ibm.jsse2.util.g.a(g.java:3)
      at com.ibm.jsse2.pc.a(pc.java:11)
      at com.ibm.jsse2.pc.checkServerTrusted(pc.java:18)
      at com.ibm.jsse2.pc.b(pc.java:56)
      at com.ibm.jsse2.lb.a(lb.java:602)
      ... 70 more
Caused by:
java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
      java.security.cert.CertPathValidatorException: The certificate issued by CN=******* Group Internal CA V1 is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.ibm.security.cert.PKIXCertPathBuilderImpl.engineBuild(PKIXCertPathBuilderImpl.java:411)
      at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:258)
      at com.ibm.jsse2.util.h.b(h.java:108)
      ... 76 more
Caused by:
java.security.cert.CertPathValidatorException: The certificate issued by CN=****** Group Internal CA V1 is not trusted; internal cause is:
      java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.ibm.security.cert.BasicChecker.<init>(BasicChecker.java:111)
      at com.ibm.security.cert.PKIXCertPathValidatorImpl.engineValidate(PKIXCertPathValidatorImpl.java:178)
      at com.ibm.security.cert.PKIXCertPathBuilderImpl.myValidator(PKIXCertPathBuilderImpl.java:737)
      at com.ibm.security.cert.PKIXCertPathBuilderImpl.buildCertPath(PKIXCertPathBuilderImpl.java:649)
      at com.ibm.security.cert.PKIXCertPathBuilderImpl.engineBuild(PKIXCertPathBuilderImpl.java:357)
      ... 78 more
Caused by:
java.security.cert.CertPathValidatorException: Certificate chaining error
      at com.ibm.security.cert.CertPathUtil.findIssuer(CertPathUtil.java:298)
      at com.ibm.security.cert.BasicChecker.<init>(BasicChecker.java:108)
      ... 82 more
INFO|2016-10-08 09:28:02,217|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  Transaction=Generic value for class com.mbb.dp3.pin.MBBPin ObjectId = null
INFO|2016-10-08 09:28:02,217|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  DrawerType={}
INFO|2016-10-08 09:28:02,217|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  Transaction=null
INFO|2016-10-08 09:28:02,217|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  Transaction=null
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  metaTransaction=com.s1.fst.transaction.meta.object.ConcreteTransaction@26752675
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  workflowName=dp3.mbbmykadpostworkflow
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>reverseMode>>>>>null
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>reenterMode>>>>>null
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>validateProcessTransaction>>>>>>Online
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>validateProcessTransaction>>isOnline4ServerSideUseOnly>>>>>>true
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137:  postTransactionResponse: return processTransaction(transaction, drawerType, parameters, workflowName);
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  transaction=Generic value for class com.mbb.dp3.pin.MBBPin ObjectId = null
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  drawerType=null
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  parameters={}
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  workflowName=dp3.mbbmykadpostworkflow
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||CALLING setDataForDynaTraceTag....
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  localUserService=com.s1.fst.app.user.ServerLocalUserService@2cc22cc2
INFO|2016-10-08 09:28:02,218|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  currentStatus=memo
INFO|2016-10-08 09:28:02,229|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  parameters.put()={transaction=Generic value for class com.mbb.dp3.pin.MBBPin ObjectId = 345740617}
INFO|2016-10-08 09:28:02,229|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||OEM137: processTransaction.  getTransactionManager()= Null
INFO|2016-10-08 09:28:02,242|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||UsingEhCache>>>>>>> Object found with id == 36557, Name == CLK-BR-SGC
INFO|2016-10-08 09:28:02,243|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||nullisHostOverride >> 
INFO|2016-10-08 09:28:02,243|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>>>>>> MBBJournalStoreJournalEntryStrategy>>>>>>>>>>com.s1.fst.domain.journal.JournalEntryValue ID = null
INFO|2016-10-08 09:28:02,243|BCHS1APPRD01|java.lang.Runtime@1c201c2|WebContainer : 0|00090506|||>>>>>>>>>> MBBJournalStoreJournalEntryStrategy>>>>>>>>>>Supervisor Override required for non-STP Pin Maintenenace
Comment
Watch Question
CERTIFIED EXPERT
Commented:
This problem has been solved!
Unlock 1 Answer and 3 Comments.
See Answer
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE