Solved

Unable to connect to the MKS in vSphere are rebooting ESXi 4.1 host

Posted on 2013-06-08
16
1,600 Views
Last Modified: 2013-07-07
After rebooting an ESXI host, I'm not longer able to view the console sessions in the Console tabs!

What I get is a message bar at the top that states:

"Unable to connect to the MKS:  Failed to connect to the server XX.XX.XX.XX:902"

I've Googled this that all I can find is suggestions on modifying the host file and modifying firewall settings, but I can replicate on the same network segment and on different laptops with different users.

There are a number of virtual servers on this host with all but one working as expected outside of this, except one, which is the file server, has stopped publishing network shares, which it gets from ISCSI mappings via the ESXI host.

On typing this, there may be one thing that I have changed since the reboot, which oddly enough was to disconnect the physical KVM from it.  I find it hard to believe this could be the culprit, but I thought I'd add this point just in case someone else has come up against this issue and MKS does stand for Mouse, Keyboard, Screen after all!!  By the way, I'm working remote form it at the moment, but can go in if need be.

Urgent assistance would really be appreciated if anyone picks this one up.

Kind Regards

Lee
0
Comment
Question by:FSIFM
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 9
  • 7
16 Comments
 
LVL 121
ID: 39232170
This can be a number of reasons, access to port 902.

1. any firewalls changed on your network?

2. does this happen to all Virtual Machines?

3. can you telnet to <ip address of host> 902

4. can you telnet to <dns or hostname of host> 902

5. repeat 3 & 4 above with ports 80 and 443

6. can you ping by IP Address and hostname

7. can you traceroute by IP address and hostname e.g. reverse DNS

8. there is no router, internet or NAT involved? e.g. is this local LAN

9. Restart Network Management Agents on Host

10. try restarting

11. un-register one of the VMs, and re-register with inventory.
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39232199
1. any firewalls changed on your network?  
No, but despite them being rebooted, giving the possibility of a un-saved config, I can replicate this on the same network segment with no firewalls in between.

2. does this happen to all Virtual Machines?  
Only on this esxi host yes.  Another 4.1.0 esxi host does not have this issue after being rebooted.

3. can you telnet to <ip address of host> 902
No

4. can you telnet to <dns or hostname of host> 902
No

5. repeat 3 & 4 above with ports 80 and 443
Yes I can telnet on these ports.

6. can you ping by IP Address and hostname
Yes

7. can you traceroute by IP address and hostname e.g. reverse DNS
Yes

8. there is no router, internet or NAT involved? e.g. is this local LAN
Yes its local LAN

9. Restart Network Management Agents on Host
Done, but no luck.

10. try restarting
Done many times. 8-(

11. un-register one of the VMs, and re-register with inventory.
No joy on this one either.
 
I'm in the office now and have connected the physical KVM and rebooted just in case there is an odd bug, but no luck on that one either.
0
 
LVL 121
ID: 39232205
You should get a connection to 902...

something like the following should respond:-

220 VMware Authentication Daemon Version 1.10: SSL Required, ServerDaemonProtoco
l:SOAP, MKSDisplayProtocol:VNC , VMXARGS supported

tried a /sbin/services.sh restart from the console or ssh
0
Connect further...control easier

With the ATEN CE624, you can now enjoy a high-quality visual experience powered by HDBaseT technology and the convenience of a single Cat6 cable to transmit uncompressed video with zero latency and multi-streaming for dual-view applications where remote access is required.

 
LVL 4

Author Comment

by:FSIFM
ID: 39232222
I'm getting a connection refused when I try to create a SSH connection (via Putty), but I can confirm that Remote Tech Support (SSH) service is running (via the vSphere Client).

From the vSphere client, I have manually restarted the services, but that hasn't helped either.  is this that same thing as running this command?

I can telnet on port 902 to another 4.1 esxi host, so its something on this particular host that blocking it as its all on the same network segment of our LAN.
0
 
LVL 121
ID: 39232233
firewalls will block tcp 902

or, ESXi 4.1 host is broke.

no, services.sh, restarts ALL the services.
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39232242
Definitely no firewalls in between.  I can replicate from a server (again on same network segment) that doesn't have a local firewall on.

ESXi host being broke is looking like a possibility, is there a way to confirm this.

As I cant connect via putty, I've pressed ALT + F1 at the console login screen to enter the hidden Console, but the command doesn't run from the root.  Can you remind me the syntax that I need to run to get to the folder that the command can run from?
0
 
LVL 121
ID: 39232244
can you login? and get a console prompt?

 /sbin/services.sh restart

the above does not work?

login as: root
Using keyboard-interactive authentication.
Password:
The time and date of this login have been sent to the system logs.

VMware offers supported, powerful system administration tools.  Please
see www.vmware.com/go/sysadmintools for details.

The ESXi Shell can be disabled by an administrative user. See the
vSphere Security documentation for more information.
~ # cd /sbin
/sbin # ls services.sh
services.sh
/sbin # services.sh
Usage: services.sh {start|stop|restart}
/sbin #

Open in new window

0
 
LVL 4

Author Comment

by:FSIFM
ID: 39232248
Oops, must have been a typo, yes it does work from the console prompt, but its still not fixed it.
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39232288
Thanks for you advice so far Hanccocka.

With the exception of rebuilding the host, can you think of any thing else?  I do have a server scheduled to replace it, but ideally, the replacement cant happen until 26th June.

Another thing that may be relevant is that we have a Dell MD 3000i hanging off the host as well (via 2 virtual switches), and one virtual server that utilises it.  Out of all the virtual servers on the host, this seems to the only one that doesn't work!!  Perhaps a little more serious than the ESXi issue, but I'm hoping the issue is related, hence I'm focussing on ESXi first.

I've checked that the iSCSI cables are in the correct VLANs etc., so if you have any further ideas, your input will be appreciated.
0
 
LVL 121

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 39232315
something is wrong with services, it's quicker to re-install (only takes 7mins).

and after re-install, it should not take long to re-configure 30mins.
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39233437
I was thinking of doing an in place upgrade to 5.1.  Can you see any issues when that?

As a precaution, I'm trying to copy some of the key servers from the datastore on to my local disk, but I'm getting "Failed to log into nfc server" error.  According to a post I found (http://communities.vmware.com/message/857754#857754), this also suggests an issue with port 902!  The suggestions in this didn't help either. 8-(  Haven't tried connecting directly to the ESXi server yet thought, but that's not exactly a fix.
0
 
LVL 121
ID: 39233471
If I had an issue with a server, I would not do an in-place upgrade, you could carry the issues forward.

HOW TO: Connect to the VMware vSphere Hypervisor 5.1 (ESXi 5.1) using the vSphere Client

Re-installation can maintain the VMs on the VMFS datastore, if selected.

Also check your server is supported and on the HCL, requirements have changed for 5.1.

Check the VMware HCL (Hardware Compatability Lists) here

http://www.vmware.com/go/hcl

http://www.vmware.com/resources/compatibility/search.php
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39233499
Bit worried now!  There's a few virtual servers on the System drive, so I tested moving a few ISOs between this and any of the other 2 local datastores and it comes back saying "Error caused by file   /vmfs/volumes/..."

If I boot from a 5.1 ISO and install over the top, will it format the system drive?  I cant afford to lose the virtual servers that reside on this drive.
0
 
LVL 121
ID: 39233939
No, it will ask the question preserve VMFS (e.g. VMs on dataastore), select Yes.
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39244993
Just to keep you posted, I've got the ESXi 5.1 media ready, but need to plan the install as its got a number of production servers on this box which mean I need to coordinate some downtime.
0
 
LVL 4

Author Closing Comment

by:FSIFM
ID: 39305709
I didn't exactly re-install, but instead I installed ESXi 5.1 to a USB stick on an identical server and booted from it on the problematic server.  This picked up the datastores and I was able to copy files to and from them.  I could also view the console as expected without getting the MKS error.
0

Featured Post

On Demand Webinar: Networking for the Cloud Era

Did you know SD-WANs can improve network connectivity? Check out this webinar to learn how an SD-WAN simplified, one-click tool can help you migrate and manage data in the cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

HOW TO: Upload an ISO image to a VMware datastore for use with VMware vSphere Hypervisor 6.5 (ESXi 6.5) using the vSphere Host Client, and checking its MD5 checksum signature is correct.  It's a good idea to compare checksums, because many installat…
Giving access to ESXi shell console is always an issue for IT departments to other Teams, or Projects. We need to find a way so that teams can use ESXTOP for their POCs, or tests without giving them the access to ESXi host shell console with a root …
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
Teach the user how to join ESXi hosts to Active Directory domains Open vSphere Client: Join ESXi host to AD domain: Verify ESXi computer account in AD: Configure permissions for domain user in ESXi: Test domain user login to ESXi host:
Suggested Courses

632 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question