vSphere Upgrade Saga: vSphere 6.0

Having upgraded all the management tools that make up vCloud Suite 6.0, it became time to upgrade vSphere itself to 6.0. My upgrade did not go as smoothly as I had expected. Following are the steps I took for the upgrade, plus an account of the one major problem I had and its solution.

The steps to upgrade vSphere to a major revision require using either VMware Upgrade Manager (VUM), or VUM  plus Auto Deploy. In either case, you need to do the following to at least one host:

  • Import your distribution ISO into VUM.
  • Use VMware Update Manager to upgrade a host.
  • Ensure settings are as you require.
  • Update or create a new host profile for the release from the newly minted system.
  • For Auto Deploy, you need to make a master image.
  • Deploy using VUM or Auto Deploy the rest of the nodes in each cluster.
  • Apply the new host profile.

This is a pretty straightforward set of steps, ones that I have been doing for years and since ESX v2.x in some fashion. Yet, this upgrade had a major issue, which thankfully has a simple solution—not an ideal solution, but there is one. The problem:

After I upgraded the first node, two of my virtual distributed switch (VDS) port groups went offline. Mind you, not all my port groups went offline, just one on each of my VDSes with three or more port groups.

This was a not a good state of affairs, as during this I lost access to my Microsoft Windows VUM server. The quick solution was to move all my management virtual machines to a virtual standard switch (VSS) I always have available. Actually, vCenter lives on this VSS due to other past issues with non-ephemeral VDS port groups.

The real impact was to my Horizon View infrastructure. Those VMs could not be moved to a VSS very easily, as I would have to break linkages to uplinks that kept the other part of my environment running. The problem was with only one port group on each VDS. The solution for my VDS issue was to enable IPV6 per KB Article #2117308 (thank you, VMware Support, for pointing this one out to me).

Granted, before I found this solution, I tried to recreate the port group, looked through all the settings, and then used Log Insight to look at the latest messages. This pointed me to another problem, of NoneZ appearing instead of a timestamp (fixed with KB Article #2111202). In looking at the logs, I realized that Log Insight does not organize things very well on the “last five minutes” view. It only brings up the messages it thinks you should see, which in this case were not related to the issue. Instead, I had to filter specifically on VMkernel on a specific node, and then I saw the errors I needed to find the fix.

After that, I did the following:

  • Upgraded all VDSes.
  • Patched the hosts to the latest firmware and vSphere patches since the 6.0 release.
  • Upgraded VSAN using KB Article #2113221.
  • Recaptured and applied the host profile; this required me to play around with my host profile storage settings once more.
  • Upgraded VMware Tools (actually, I started doing this before I realized there were patches and had to do it once more for all VMs).
  • Upgraded my Horizon View master VM with all Microsoft and software patches, new VMware Tools, and new View agents. I also had to include the steps from KB Article #2059786, as RDP-style connections from within my network stopped working due to RDP changes in Windows 8.1 desktops.
  • Recomposed all my Horizon View virtual desktops, which I accomplished by committing my snapshot made above and reprovisioning the desktops automatically. I tried a recompose, but it never picked up all the changes. I rebuilt my desktops several times; luckily, I never had much on my personal disks, so I did not mind those going away.

Now my environment is sitting at vCloud Suite 6.0 and seems to be working quite smoothly. All in all, outside of the VDS issue, the upgrade also went smoothly. Solving that issue, however, requires a better understanding of Log Insight. Now that I know how to use this important tool better, I can build out some new rules and content packs.

Edward Haletky

Edward L. Haletky, aka Texiwill, is an author, analyst, developer, technologist, and business owner. Edward owns AstroArch Consulting, Inc., providing virtualization, security, network consulting and development and TVP Strategy where he is also an Analyst. Edward is the Moderator and Host of the Virtualization Security Podcast as well as a guru and moderator for the VMware Communities Forums, providing answers to security and configuration questions. Edward is working on new books on Virtualization.

Leave a Reply

Your email address will not be published. Required fields are marked *

1 × 1 =