My upgrade from ESXi 6.5 to 6.7 did not go as cleanly as I would have liked, for several reasons. However, so far I have found a solution to everything. Here is a recap of my approach.
First, I have HPE BL460c Gen8 servers. The Gen8 servers have no pre-created HPE ESXI 6.7 image. Installing the Gen9 version may cause issues, from what I have read. So, I chose to upgrade not using the HPE-specific image but using the default VMware image.
Step 1: Read the image into VMware Upgrade Manager
This step went flawlessly. It is the easiest way to do an upgrade.
Step 2: Upgrade a node
The upgrade worked with no problems. I should note that I have three-node VSAN install, and evacuating each node does not always work. I also have drivers supposedly not on the HCL for vSphere. They work just fine. The HPE Gen8 servers will have drivers not listed on the HCL, as HPE only supports the last two versions of hardware on the HCL officially. However, the initial upgrade hit no snags.
Step 3: Upgrade to latest
This step failed due to an “esxupdate error codes: -1.” I found a helpful post that talked about a similar problem. To fix this, I put the host in maintenance mode and followed the instructions. The VMA replacement I built means I don’t need to log in to ESXi directly but can run esxcli using my vCenter credentials.
However, I did not reboot as the article says. I instead upgraded my node to the latest. This hit the dreaded VSAN health check error. I followed KB2151813 to alleviate these issues. I silenced the following alerts in order to proceed:
- controlleronhcl
- controllerdriver
- controllerreleasesupport
- controllerfirmware
- vendortoolpresence
- upgradelowerhosts
- hostmissing
- collection
Once I silenced the above alarms, I was able not only to upgrade, but also to upgrade all other hosts. Interestingly enough, the esxupdate error codes: -1 appeared at different times during the install for each node (see Step 5 below).
Step 4 (maybe during 3): Clear browser cache
However, the HTML5 client continued to error with:
Response with status: 401 OK for URL: https://vcenter/ui/vum-ui/rest/vcobjects/urn:vmomi:HostSystem:host-???/isUpdateSupported
This error forced me to upgrade my entire cluster using the Flash UI. Not something I wished to do.
Clearing the cache and cookies of my browser fixed this issue. However, it seemed I had to clear the cache and cookies to use the HTML5 VUM client after a timeout or logout.
Step 5: Upgrade HPE-specific items
These upgrades are part of the noncritical host patches in some way. I did not know VMware had added vendor-specific patches into the predefined settings. I had to once more disable the VSAN alerts before remediation, then remediate each host in turn. The esxupdate error codes: -1 appeared on two hosts at this step. See Step 1 for remediation steps.
Step 6: Undo silenced alarms
After the upgrade, I unsilenced the following alerts, as I know that the HCL will have issues with my versions of drivers and firmware (at least until I can update the firmware on every node and reverify). Unfortunately, since I use Gen8 hardware, I do not think this will ever show correctly. I unsilenced the following:
- vendortoolpresence
- upgradelowerhosts
- hostmissing
- collection
Conclusion
Now my cluster is at version 6.7.0 Update1. However, that is not the end of the upgrades. I still have to upgrade VSAN, VMware Tools, Virtual Hardware, etc. Upgrading to 6.7 was a bit more massive an undertaking than other upgrades had been.
