<

Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x

vCenter Troubleshooting TIPS: ESXi 6.0 reports “error code 99” during Stage patches to entity VUM operation

Published on
9,498 Points
6,498 Views
Last Modified:
Luciano Patrão
My main areas of expertise are VMware, Virtual Backups and Storage design.
I am a vExpert and a Technical Blogger in the same areas.
This is similar to the error described in my previous Article, but with a different source problem and a different solution. When trying to scan and stage the ESXi 6.0 updates using VMware Update Manager, we can get "error code 90"
The configuration is an Hewlett-Packard DL360 G7 with the ESXi 6.0 build 3620759.
 
When trying to use the VMware update manager in this host, the scan runs without any problem, but when trying to stage the updates, we can get:

Stage patches to entity esxi01.localdomain. The host returns esxupdate error code:99.
An unhandled exception was encountered. Check the Update Manager log files and esxupdate log files for more details.
Looking at the esxpudate.log (in ESXi var/log/) I see lot of failed tries to upload VIB files to the repository, along with this:
 


Open in new window

image.gifimage.gif
So we need to check the Host Software profile(not the same as Host Profile) and check the issue.
 

Then I notice that somehow the ESXi lost its software profile, so it was not possible to stage any update, or even install a VIB manually. When troubleshooting the issue, I found out that there was no software profile in this host.
 

root@esxi01:~] esxcli software profile get
Exception]
No host image profile defined
Please refer to the log file for more details.

Open in new window


Also checking the VIB list in this host I get this:

[root@esxi01:~] esxcli software vib list

Name         Version             Vendor  Acceptance Level  Install Date
-----------  ------------------  ------  ----------------  ------------
tools-light  6.0.0-2.34.3620759  VMware  VMwareCertified   2016-04-23

Open in new window


So we need to check the VIB folder and profiles folder in /var/db/esximg and I noticed the two folders were empty.

[root@esx01:~] ls -l /var/db/esximg/*
/var/db/esximg/profiles:
total 0

/var/db/esximg/vibs:
total 0
[root@esx01:~]

Open in new window


When checking the same information from a working ESXi host I get this:

[root@esxi02:~] ls -l /var/db/esximg/profiles/
total 24
-r--r--r--    1 root     root         23998 Apr 23 12:05 %28Updated%29%20HP-ESXi-6.0.0-iso-600.9.1.39-687127855

[root@esxi02:~] esxcli software profile get
(Updated) HP-ESXi-6.0.0-iso-600.9.1.39
   Name: (Updated) HP-ESXi-6.0.0-iso-600.9.1.39
   Vendor: esxi02.localdomain
   Creation Time: 2016-04-23T12:05:31
   Modification Time: 2016-04-23T12:05:31
   Stateless Ready: True
   Description:

      (Original Vendor):Hewlett-Packard
      2016-04-23T12:05:30.688280+00:00: The following VIBs are
      installed:
        hpssacli      2.40.13.0-6.0.0.1854445
        hponcfg       6.0.0.04-00.14.4.2494585
        conrep        6.0.0.01-01.00.7.2494585
        hptestevent   6.0.0.01-01.00.5.2494585
        hpbootcfg     6.0.0.02-02.00.6.2494585

Open in new window


And this:

[root@esxi02:~] esxcli software vib list
Name                           Version                         Vendor           Acceptance Level  Install Date
----------------------  -------------------------------------  ---------------  ----------------  ------------
net-tg3                 3.137l.v60.1-1OEM.600.0.0.2494585      BRCM             VMwareCertified   2016-01-23
elxnet                  10.7.110.13-1OEM.600.0.0.2768847       EMU              VMwareCertified   2016-04-23
ima-be2iscsi            10.7.110.10-1OEM.600.0.0.2159203       EMU              VMwareCertified   2016-04-23
lpfc                    10.7.110.4-1OEM.600.0.0.2768847        EMU              VMwareCertified   2016-04-23
scsi-be2iscsi           10.7.110.10-1OEM.600.0.0.2159203       EMU              VMwareCertified   2016-04-23
amsHelper               600.10.4.0-22.2494585                  HPE              PartnerSupported  2016-04-23
....
....
scsi-megaraid-mbox      2.20.5.1-6vmw.600.0.0.2494585          VMware           VMwareCertified   2015-08-23
scsi-megaraid-sas       6.603.55.00-2vmw.600.0.0.2494585       VMware           VMwareCertified   2015-08-23
scsi-megaraid2          2.00.4-9vmw.600.0.0.2494585            VMware           VMwareCertified   2015-08-23
scsi-mptsas             4.23.01.00-9vmw.600.0.0.2494585        VMware           VMwareCertified   2015-08-23
scsi-mptspi             4.23.01.00-9vmw.600.0.0.2494585        VMware           VMwareCertified   2015-08-23
uhci-usb-uhci           1.0-3vmw.600.0.0.2494585               VMware           VMwareCertified   2015-08-23
vsan                    6.0.0-2.34.3563498                     VMware           VMwareCertified   2016-04-23
vsanhealth              6.0.0-3000000.3.0.2.34.3544323         VMware           VMwareCertified   2016-04-23
xhci-xhci               1.0-3vmw.600.2.34.3620759              VMware           VMwareCertified   2016-04-23
tools-light             6.0.0-2.34.3620759                     VMware           VMwareCertified   2016-04-23
scsi-qla2xxx            911.k1.1-26OEM.500.0.0.472560          qlogic           VMwareCertified   2015-08-23

Open in new window


Searching in the Internet for the same type of issue, I did not find anything similar. The VMware KnowledgeBase has a solution for the “error code 99” by Initializing VUM without reinstall, but since this is a problem in one host, this could not be the solution for this particular issue.
 

I then found an article with a similar issue in Patrick Terlisten blog. Since I didn't want to reinstall the ESXi host, I needed to try this solution, even if was not 100% identical, it was almost the same.
 

I started by following Patrick's recommendation to copy the esximg files/folders from a working host (in this case esxi-02) to the corrupted ESXi host (esxi-01).
 

First if  you don't have your SSH Client enabled in the host firewall, you need to set it enabled to do the next task using the SCP command. To enable SSH Client in the source ESXi host:

[root@esx02:~] esxcli network firewall ruleset set --enabled true --ruleset-id=sshClient

Open in new window


Note: Don't forget to disable SSH Client after do this tasks.
 

After you run the SCP command you will be prompted for the root password of the remote host and once you have successfully authenticated the file will copy.

[root@esxi02:~] scp -r /var/db/esximg/ root@esxi02:/var/db
The authenticity of host 'esxi02 (esxi02)' can't be established.
RSA key fingerprint is SHA256:isiF8md0Q6GDazZ97fbJ/4ZiqxOrf9tE4mHv0XN64kM.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'esxi02' (RSA) to the list of known hosts.
Password:
sata-sata-sil-1748273158.xml                                   100% 1717     1.7KB/s   00:00
lsi-mr3-989864457.xml                                          100% 1504     1.5KB/s   00:00
scsi-ips--1979861494.xml                                       100% 1619     1.6KB/s   00:00
char-hpcru--1874046437.xml                                     100% 1638     1.6KB/s   00:00
net-tg3--917722591.xml                                         100% 1461     1.4KB/s   00:00
ipmi-ipmi-devintf-1862766627.xml                               100% 1719     1.7KB/s   00:00
sata-sata-promise--860290007.xml                               100% 1742     1.7KB/s   00:00
esx-ui--629271967.xml                                          100% 53KB  53.2KB/s   00:00
cpu-microcode-1845960806.xml                                   100% 2381     2.3KB/s   00:00
scsi-bnx2i--630528917.xml                                      100% 1407     1.4KB/s   00:00
scsi-fnic-248026735.xml                                        100% 1784     1.7KB/s   00:00
scsi-hpdsa--570772364.xml                                      100% 1412     1.4KB/s   00:00
qlnativefc--775726474.xml                                      100% 1408     1.4KB/s   00:00
hpe-ilo--120862598.xml                                         100% 1550     1.5KB/s   00:00
net-cnic-1577337473.xml                                        100% 1389     1.4KB/s   00:00
scsi-qla4xxx-1755174022.xml                                    100% 1517     1.5KB/s   00:00
hpe-smx-provider-1463103112.xml                                100% 5426     5.3KB/s   00:00
...
...
net-bnx2x-272405923.xml                                        100% 1534     1.5KB/s   00:00
net-bnx2-984265687.xml                                         100% 1507     1.5KB/s   00:00
net-ixgbe--494567451.xml                                       100% 1415     1.4KB/s   00:00
scsi-mpt2sas--1969732120.xml                                   100% 1679     1.6KB/s   00:00
scsi-megaraid-mbox--26860043.xml                               100% 1775     1.7KB/s   00:00
scsi-aic79xx-757558775.xml                                     100% 1643     1.6KB/s   00:00
%28Updated%29%20HP-ESXi-6.0.0-iso-600.9.1.39-687127855         100% 23KB  23.4KB/s   00:00
[root@esxi02:~]

Open in new window


After copying the files, just check if the files are in the ESXi host (esxi01).
 

Host Software profile:

[root@esx01:~] esxcli software profile get
(Updated) HP-ESXi-6.0.0-iso-600.9.1.39
   Name: (Updated) HP-ESXi-6.0.0-iso-600.9.1.39
   Vendor: deacvm-dsc-002
   Creation Time: 2016-04-23T12:05:29
   Modification Time: 2016-04-23T12:05:29
   Stateless Ready: False
   Description:

      (Original Vendor):Hewlett-Packard
      2016-04-23T12:05:29.506598+00:00: The following VIBs are
      installed:
        hpssacli      2.40.13.0-6.0.0.1854445
        hponcfg       6.0.0.04-00.14.4.2494585
        conrep        6.0.0.01-01.00.7.2494585
        hptestevent   6.0.0.01-01.00.5.2494585
        hpbootcfg     6.0.0.02-02.00.6.2494585


[root@esx01:~] ls -l /var/db/esximg/profiles/
total 24
-r--r--r--    1 root     root         23998 Apr 23 12:05 %28Updated%29%20HP-ESXi-6.0.0-iso-600.9.1.39-687127855
[root@esx01:~]

Open in new window


After checking that the files are copied into the destination, run the VUM Scan again on the host before and after staging to ensure no errors are found. After that I was able to remediate the ESXi host and fully update it.
 

Again, thanks to Patrick for his solution and for putting me on the right track.
 

Hope this article can you help fixing this error that you may encounter in your ESXi upgrade.

This is the part of my "TIP Articles". So, please vote "Helpful" on this Article. And I encourage your comments and feedback.
0
Comment
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
2 Comments
 

Expert Comment

by:Kenneth Bolt
Does this copy affect the running VMs ?
0
 

Expert Comment

by:Peter Navarro
Worked perfectly.
0

Featured Post

Put Machine Learning to Work--Protect Your Clients

Machine learning means Smarter Cybersecurity™ Solutions.
As technology continues to advance, managing and analyzing massive data sets just can’t be accomplished by humans alone. It requires huge amounts of memory and storage, as well as the high-speed power of the cloud.

Join & Write a Comment

Teach the user how to use create log bundles for vCenter Server or ESXi hosts Open vSphere Web Client: Generate vCenter Server and ESXi host log bundle:  Open vCenter Server Appliance Web Management interface and generate log bundle: Open vCenter Se…
This video shows you how easy it is to boot from ISO images for virtual machines with the ISO images stored on a local datastore on the ESXi host.

Keep in touch with Experts Exchange

Tech news and trends delivered to your inbox every month