Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Unable to connect to the MKS in vSphere are rebooting ESXi 4.1 host

Posted on 2013-06-08
16
Medium Priority
?
1,628 Views
Last Modified: 2013-07-07
After rebooting an ESXI host, I'm not longer able to view the console sessions in the Console tabs!

What I get is a message bar at the top that states:

"Unable to connect to the MKS:  Failed to connect to the server XX.XX.XX.XX:902"

I've Googled this that all I can find is suggestions on modifying the host file and modifying firewall settings, but I can replicate on the same network segment and on different laptops with different users.

There are a number of virtual servers on this host with all but one working as expected outside of this, except one, which is the file server, has stopped publishing network shares, which it gets from ISCSI mappings via the ESXI host.

On typing this, there may be one thing that I have changed since the reboot, which oddly enough was to disconnect the physical KVM from it.  I find it hard to believe this could be the culprit, but I thought I'd add this point just in case someone else has come up against this issue and MKS does stand for Mouse, Keyboard, Screen after all!!  By the way, I'm working remote form it at the moment, but can go in if need be.

Urgent assistance would really be appreciated if anyone picks this one up.

Kind Regards

Lee
0
Comment
Question by:FSIFM
  • 9
  • 7
16 Comments
 
LVL 124
ID: 39232170
This can be a number of reasons, access to port 902.

1. any firewalls changed on your network?

2. does this happen to all Virtual Machines?

3. can you telnet to <ip address of host> 902

4. can you telnet to <dns or hostname of host> 902

5. repeat 3 & 4 above with ports 80 and 443

6. can you ping by IP Address and hostname

7. can you traceroute by IP address and hostname e.g. reverse DNS

8. there is no router, internet or NAT involved? e.g. is this local LAN

9. Restart Network Management Agents on Host

10. try restarting

11. un-register one of the VMs, and re-register with inventory.
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39232199
1. any firewalls changed on your network?  
No, but despite them being rebooted, giving the possibility of a un-saved config, I can replicate this on the same network segment with no firewalls in between.

2. does this happen to all Virtual Machines?  
Only on this esxi host yes.  Another 4.1.0 esxi host does not have this issue after being rebooted.

3. can you telnet to <ip address of host> 902
No

4. can you telnet to <dns or hostname of host> 902
No

5. repeat 3 & 4 above with ports 80 and 443
Yes I can telnet on these ports.

6. can you ping by IP Address and hostname
Yes

7. can you traceroute by IP address and hostname e.g. reverse DNS
Yes

8. there is no router, internet or NAT involved? e.g. is this local LAN
Yes its local LAN

9. Restart Network Management Agents on Host
Done, but no luck.

10. try restarting
Done many times. 8-(

11. un-register one of the VMs, and re-register with inventory.
No joy on this one either.
 
I'm in the office now and have connected the physical KVM and rebooted just in case there is an odd bug, but no luck on that one either.
0
 
LVL 124
ID: 39232205
You should get a connection to 902...

something like the following should respond:-

220 VMware Authentication Daemon Version 1.10: SSL Required, ServerDaemonProtoco
l:SOAP, MKSDisplayProtocol:VNC , VMXARGS supported

tried a /sbin/services.sh restart from the console or ssh
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
LVL 4

Author Comment

by:FSIFM
ID: 39232222
I'm getting a connection refused when I try to create a SSH connection (via Putty), but I can confirm that Remote Tech Support (SSH) service is running (via the vSphere Client).

From the vSphere client, I have manually restarted the services, but that hasn't helped either.  is this that same thing as running this command?

I can telnet on port 902 to another 4.1 esxi host, so its something on this particular host that blocking it as its all on the same network segment of our LAN.
0
 
LVL 124
ID: 39232233
firewalls will block tcp 902

or, ESXi 4.1 host is broke.

no, services.sh, restarts ALL the services.
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39232242
Definitely no firewalls in between.  I can replicate from a server (again on same network segment) that doesn't have a local firewall on.

ESXi host being broke is looking like a possibility, is there a way to confirm this.

As I cant connect via putty, I've pressed ALT + F1 at the console login screen to enter the hidden Console, but the command doesn't run from the root.  Can you remind me the syntax that I need to run to get to the folder that the command can run from?
0
 
LVL 124
ID: 39232244
can you login? and get a console prompt?

 /sbin/services.sh restart

the above does not work?

login as: root
Using keyboard-interactive authentication.
Password:
The time and date of this login have been sent to the system logs.

VMware offers supported, powerful system administration tools.  Please
see www.vmware.com/go/sysadmintools for details.

The ESXi Shell can be disabled by an administrative user. See the
vSphere Security documentation for more information.
~ # cd /sbin
/sbin # ls services.sh
services.sh
/sbin # services.sh
Usage: services.sh {start|stop|restart}
/sbin #

Open in new window

0
 
LVL 4

Author Comment

by:FSIFM
ID: 39232248
Oops, must have been a typo, yes it does work from the console prompt, but its still not fixed it.
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39232288
Thanks for you advice so far Hanccocka.

With the exception of rebuilding the host, can you think of any thing else?  I do have a server scheduled to replace it, but ideally, the replacement cant happen until 26th June.

Another thing that may be relevant is that we have a Dell MD 3000i hanging off the host as well (via 2 virtual switches), and one virtual server that utilises it.  Out of all the virtual servers on the host, this seems to the only one that doesn't work!!  Perhaps a little more serious than the ESXi issue, but I'm hoping the issue is related, hence I'm focussing on ESXi first.

I've checked that the iSCSI cables are in the correct VLANs etc., so if you have any further ideas, your input will be appreciated.
0
 
LVL 124

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 2000 total points
ID: 39232315
something is wrong with services, it's quicker to re-install (only takes 7mins).

and after re-install, it should not take long to re-configure 30mins.
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39233437
I was thinking of doing an in place upgrade to 5.1.  Can you see any issues when that?

As a precaution, I'm trying to copy some of the key servers from the datastore on to my local disk, but I'm getting "Failed to log into nfc server" error.  According to a post I found (http://communities.vmware.com/message/857754#857754), this also suggests an issue with port 902!  The suggestions in this didn't help either. 8-(  Haven't tried connecting directly to the ESXi server yet thought, but that's not exactly a fix.
0
 
LVL 124
ID: 39233471
If I had an issue with a server, I would not do an in-place upgrade, you could carry the issues forward.

HOW TO: Connect to the VMware vSphere Hypervisor 5.1 (ESXi 5.1) using the vSphere Client

Re-installation can maintain the VMs on the VMFS datastore, if selected.

Also check your server is supported and on the HCL, requirements have changed for 5.1.

Check the VMware HCL (Hardware Compatability Lists) here

http://www.vmware.com/go/hcl

http://www.vmware.com/resources/compatibility/search.php
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39233499
Bit worried now!  There's a few virtual servers on the System drive, so I tested moving a few ISOs between this and any of the other 2 local datastores and it comes back saying "Error caused by file   /vmfs/volumes/..."

If I boot from a 5.1 ISO and install over the top, will it format the system drive?  I cant afford to lose the virtual servers that reside on this drive.
0
 
LVL 124
ID: 39233939
No, it will ask the question preserve VMFS (e.g. VMs on dataastore), select Yes.
0
 
LVL 4

Author Comment

by:FSIFM
ID: 39244993
Just to keep you posted, I've got the ESXi 5.1 media ready, but need to plan the install as its got a number of production servers on this box which mean I need to coordinate some downtime.
0
 
LVL 4

Author Closing Comment

by:FSIFM
ID: 39305709
I didn't exactly re-install, but instead I installed ESXi 5.1 to a USB stick on an identical server and booted from it on the problematic server.  This picked up the datastores and I was able to copy files to and from them.  I could also view the console as expected without getting the MKS error.
0

Featured Post

Get your Conversational Ransomware Defense e‑book

This e-book gives you an insight into the ransomware threat and reviews the fundamentals of top-notch ransomware preparedness and recovery. To help you protect yourself and your organization. The initial infection may be inevitable, so the best protection is to be fully prepared.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If we need to check who deleted a Virtual Machine from our vCenter. Looking this task in logs can be painful and spend lot of time, so the best way to check this is in the vCenter DB. Just connect to vCenter DB(default DB should be VCDB and using…
HOW TO: Connect to the VMware vSphere Hypervisor 6.5 (ESXi 6.5) using the vSphere (HTML5 Web) Host Client 6.5, and perform a simple configuration task of adding a new VMFS 6 datastore.
Teach the user how to edit .vmx files to add advanced configuration options Open vSphere Web Client: Edit Settings for a VM: Choose VM Options -> Advanced: Add Configuration Parameters:
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial

963 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question