There was new firmware for my HP BladeSystem so I chose this week end to do the upgrade. Perhaps that was a mistake, but I was having connection issues to my Interconnects and reporting was not happening. I also thought it would help with my non-working small LCD screen. That was also not the case. So the saga begins innocuous enough, with a successful but odd upgrade of the OA, then onto virtual connect devices, and finally was going to end with the brocade switch upgrade, leaving the blades firmware to be upgraded next time.
The OA upgrade went fine, but it took one of my nodes offline causing an HA event to occur. But only one of the nodes, not both of them.
The Flex-10 upgrade seemed to go fine, but the new firmware did not come into play until I rebooted the Flex-10 modules and that is one everything went pear-shaped. I lost complete connectivity to the entire virtual environment within the BladeSystem. It took a few hours to find the issue, but once I did, it was easy to solve. I needed to configure Enclosure Bay IP Addressing (EBIPA) to have the proper IP addresses for the HP Flex-10 modules in order to login to the Virtual Connect Manager to complete the upgrade. Once that was done, everything worked as expected.
However, I now have another issue and one perhaps related to why the HA event occurred. Apparently I am experiencing a power condition event which is now keeping the blade from rebooting properly. A quick search of Google pointed me to a site that stated they had the same problem and had to physically remove the blade from the enclosure and in effect re-seat it. Well that worked.
Given all the issues with this upgrade, upgrading the firmware on the blades themselves and the Brocade switches will just have to wait for a few weeks. With EBIPA properly configured now it should not be such a chore.
Hi Edward.
Welcome to the world of Flex-10 upgrades which is unfortunately a convoluted process. Theoretically you should be able to upgrade Flex-10 non-disruptively but I have not been able to get this done consistently.
As you know the formware flash only takes effect when the switches are rebooted and in a stacked situation you can lose network connectivity when some of the switches are rebooted.
This is not the case if you reset the switches manually without updating firmware. You can reboot each switch in the exact same order and maintain network connectivity but when you do the same thing as part of the formware update process you can lose network so there must be some issue in the way the switches reconnect during the upgrade.
I’ve maintained a lot of information on Flex-10 on my blog so it’s worth staying up to date with all the confusion coming from HP.
I had a very similar experience last year when doing the OA/VC firmware updates. I had two stacked chassis, the VCs rebooted, and I lost ALL network connectivity to all blades in both chassis. It took several hours to restore service, a call to HP, and all of a sudden it just came back online. For this very reason I would never stack chassis, and proceed with extreme caution during non-production hours when attempting OA/VC updates. HP really needs to work out the kinks..pretty bad design.