vCenter server 2012 error 503

We have vCenter 6.5 installed on server 2012. We planned to go to 6.7 VCSA. However, the 2012 VM restarted and now vcenter web client doesnt come back up.  We get the following:


503 Service Unavailable (Failed to connect to endpoint: [class Vmacore::Http::LocalServiceSpec:0x000000deb87e6ca0] _serverNamespace = /vsphere-client action = Allow _port = 9090)

The problem we have is that there are still hosts that have 5.5 that had not been upgraded along with some distributed switches on older version that had not been updated.  What should be the route to restore access to 6.5 so we can upgrade hosts and vDS then move to 6.7 VCSA?
LVL 1
Leigh KalbliAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Murali SripadaCommented:
Can you go to c:\program files\vmware\vcenterserver\bin  from administrator command prompt
run service-control --status --all and share us the output please
0
Murali SripadaCommented:
Any recent changes on the vcenter or does the issue started post vcenter reboot?
0
Leigh KalbliAuthor Commented:
error.PNG
No changes. Just started after what appears to be a SAN got disconnected and the VM restarted.
0
The Ultimate Tool Kit for Technolgy Solution Provi

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy for valuable how-to assets including sample agreements, checklists, flowcharts, and more!

Murali SripadaCommented:
All major services are started except webclientsvc and vsphere-ui

can you start them manually using the below commands

service-control --start vsphere-ui
service-control --start vspherewebclientsvc
0
Leigh KalbliAuthor Commented:
error3.PNG
0
Murali SripadaCommented:
Please try the other command for webclientsvc as well.. if failed, please share us the log file (vsphere_client_virgo.log) from c:\programdata\vmware\vcenterserver\logs\vsphere-client\logs
0
Leigh KalbliAuthor Commented:
error4.PNG
been there for about 4 minutes.
0
Murali SripadaCommented:
Please share the log file which gives us some idea on what's happening..
0
Leigh KalbliAuthor Commented:
0
Murali SripadaCommented:
[2018-08-09T12:01:32.373-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-372' generated
[2018-08-09T12:01:32.374-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-373' generated
[2018-08-09T12:01:32.381-04:00] GiLogServiceListener@4409f6f  org.osgi.service.log.LogService                                   Bundle org.eclipse.equinox.ds_1.4.0.v20120112-1400, [SCR] Unexpected exception occurred! java.lang.IllegalStateException: BundleContext is no longer valid
      at org.eclipse.osgi.framework.internal.core.BundleContextImpl.checkValid(BundleContextImpl.java:931)
      at org.eclipse.osgi.framework.internal.core.BundleContextImpl.getServiceReferences(BundleContextImpl.java:498)
      at org.eclipse.equinox.internal.ds.Reference.hasProviders(Reference.java:127)
      at org.eclipse.equinox.internal.ds.Resolver.selectNewlyUnsatisfied(Resolver.java:600)
      at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:364)
      at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
      at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
      at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
      at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
      at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
      at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
      at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
      at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.unregister(ServiceRegistrationImpl.java:225)
      at org.eclipse.equinox.internal.region.RegionManager.stop(RegionManager.java:67)
      at org.eclipse.osgi.framework.internal.core.BundleContextImpl$2.run(BundleContextImpl.java:771)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.eclipse.osgi.framework.internal.core.BundleContextImpl.stop(BundleContextImpl.java:764)
      at org.eclipse.osgi.framework.internal.core.BundleHost.stopWorker(BundleHost.java:510)
      at org.eclipse.osgi.framework.internal.core.AbstractBundle.suspend(AbstractBundle.java:566)
      at org.eclipse.osgi.framework.internal.core.Framework.suspendBundle(Framework.java:1206)
      at org.eclipse.osgi.framework.internal.core.StartLevelManager.decFWSL(StartLevelManager.java:592)
      at org.eclipse.osgi.framework.internal.core.StartLevelManager.doSetStartLevel(StartLevelManager.java:257)
      at org.eclipse.osgi.framework.internal.core.StartLevelManager.shutdown(StartLevelManager.java:215)
      at org.eclipse.osgi.framework.internal.core.InternalSystemBundle.suspend(InternalSystemBundle.java:284)
      at org.eclipse.osgi.framework.internal.core.Framework.shutdown(Framework.java:692)
      at org.eclipse.osgi.framework.internal.core.Framework.close(Framework.java:600)
      at org.eclipse.osgi.framework.internal.core.InternalSystemBundle$1.run(InternalSystemBundle.java:261)
      at java.lang.Thread.run(Thread.java:748)

[2018-08-09T12:01:32.385-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-384' generated
[2018-08-09T12:01:32.386-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-385' generated
[2018-08-09T12:01:32.387-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-386' generated
[2018-08-09T12:01:32.420-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-419' generated
[2018-08-09T12:01:32.421-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-420' generated
[2018-08-09T12:01:32.433-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-432' generated
[2018-08-09T12:01:32.444-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-443' generated
[2018-08-09T12:01:32.466-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-465' generated
[2018-08-09T12:01:32.478-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-477' generated
[2018-08-09T12:01:32.479-04:00] [INFO ] Thread-12                     org.eclipse.virgo.medic.eventlog.default                         ME0003I Dump 'C:\PROGRA~3\VMware\VCENTE~1\logs\VSPHER~1\dump\2018-08-09-12-01-478' generated

looks like files are corrupted or config file might be corrupted related to the webclient during reboot

If you have support entitlement, please engage vmware support
0
Leigh KalbliAuthor Commented:
We do.  I will reach out to them and engage a case and post back.
0
Leigh KalbliAuthor Commented:
case opened with VM Ware and their immediate response was to restore a back up of 5.5.  Not really acceptable in this situation considering the changes that were made since it was upgraded to 6.5. And back ups control is beyond my control so arguing about having a more current back up is a dead point.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
VCSA

From novice to tech pro — start learning today.