In this article I’m going to demonstrate how to perform a VMware Cloud Foundation LCM upgrade from version 3.9.0 to 3.9.1. The bundles are:
- SDDC Manager UI patch 3.9.0.1
from 3.9.0.0 build 14866160 to 3.9.0.1 build 15219681 - SDDC all services to version 3.9.1.0
from 3.9.0.1 build 15219681 to 3.9.1.0 build 15345960 - SDDC configuration drift 3.9.1.0
from 3.9.1.0 build 15345960 to 3.9.1.0 build 15345960
Please note: that’s fine, the build number isn’t changing on the configuration drift - NSX Manager, VCF bundle version 3.10.2-119473
from NSX-V 6.4.5 build 13282012 to 6.4.6 build 14819921 - PSC/VC bundle, VCF bundle version 3.10.4-119596
from 6.7.0 build 14367737 to 6.7.0 build 15132721 - ESXi bundle, VCF bundle version 3.10.6-119471
from 6.7.0 build 14320388 (6.7 U3) to 6.7.0-15160138 (6.7 P01)
Prerequisites
All the 3.9.1 bundles have been downloaded. You can either use the lcm-bundle-transfer utility (see previous article)
or download bundles from the SDDC Manager GUI.
Remember: if you install VCF 3.9.1 as green field and later deploy vRealize Suite products they will be connected to a universal logical switch (ULS) as they’re using the Application Virtual Networks (AVN) design (see this article). On the other hand if you get to 3.9.1 by updating using SDDC Lifecycle Management (LCM) then all vRealize Suite products will be deployed on VLAN backed port groups.
Source: https://docs.vmware.com/en/VMware-Cloud-Foundation/3.9.1/rn/VMware-Cloud-Foundation-391-Release-Notes.html see Application Virtual Networks (AVNs) section
SDDC LCM Updates
From Workload Domains > View Details > select the MGMT Domain
Head over to Update/Patches and begin running Prechecks
First things first: fix any problem reported before continuing with the update. In the following example there are expired password on psc-1 so the NTP sync was failing
Since it’s a lab I went ahead and disabled password complexity on all PSCs and VCs. SSH into the PSC/VC and edit /etc/pam.d/system-password and comment out the pam_cracklib.so line:
root@vcenter-1 [ ~ ]# cat /etc/pam.d/system-password # Begin /etc/pam.d/system-password # use sha512 hash for encryption, use shadow, and try to use any previously defined # authentication token (chosen password) set by any prior module #password requisite pam_cracklib.so password required pam_unix.so sha512 shadow try_first_pass #End /etc/pam.d/system-password
Now let’s make sure that all components are in time sync.
On Photon OS you can use the following command to query the NTP time synchronisation and if not sync’ed restart the NTP client. The offset and jitter values should be close to 1 or ideally 0, anything above you might still have SDDC Manager complaining.
ntpq -p systemctl status systemd-timesyncd systemctl restart systemd-timesyncd ntpq -p
On ESXi instead the following commands apply
ntpq -p /etc/init.d/ntpd restart ntpq -p
Once that’s removed you can reset the root password to the same as before.
Let’s try again running the Precheck again to see if it’s any better. Looks like it’s all good now
We are now good to proceed with the upgrade. After a snapshot of SDDC Manager is taken we can proceed and schedule the updates.
1) 3.9.0.1 (SDDC Manager UI patch)
This is a UI fix patch that takes the system from 3.9.0.0 to 3.9.0.1
2) 3.9.1.0 (SDDC all services)
Next to go is bundle “VMware Cloud Foundation Update 3.9.1.0” which takes the system from 3.9.0.1 to 3.9.1 (build 15345960) and updates all services, so it’s going to take longer than the previous UI patch.
Whilst you’re waiting and watching paint drying I’d recommend tailing the LCM logs using:
tail -f /var/log/vmware/vcf/lcm/lcm.log
That took 47 minutes which is not bad considering my environment is nested. Once that’s done and before moving to the next bundle let’s run again the prechecks. In VCF lab vSAN is complaining about hardware compatibility and the reason this warning wasn’t there before is that I disabled the vSAN checks on application-prod.properties.
After the last LCM update all services got updated and as a result configuration files got overridden too; so I’m going to edit again the file /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties and set to false the following options (again just because my lab is nested):
3) 3.9.1.0 (SDDC configuration drift)
Run the precheck again and it’s time for the 3.9.1.0 VCF configuration drift, which takes care, as I call it, of “configuration leftover dirty bits” 🙂
So that completed the SDDC bundles, meaning all updates so were only applied to SDDC Manager and its management services. But 3.9.1 comes with other 3.9.1 bundles for the other components as well, namely NSX Manager (both V and T) PSCs, vCenters and ESXi hosts. I wrote a separate article on 3.9.1 What’s New
So the next bundle to go is NSX (V) Manager. NSX-T is the same as 3.9.0
4) NSX bundle (VCF bundle version 3.10.2-119473)
NSX Manager is updating from 6.4.5 to 6.4.6 build 14819921
5) PSC/VC bundle (VCF bundle version 3.10.4-119596)
Next is PSC/VC that are going to be updated from 6.7 U3 to 6.7 P01.
This time the update failed the first time on vCenter Server (due to latencies on my nested VC) but sub-sequentially worked in the second attempt (third screenshot) where you can see the only component listed was VC
6) ESXi bundle (VCF bundle version 3.10.6-119471)
This is the last bundle which is going to update the ESXi hosts from version 6.7 build 14320388 (6.7 U3) to 6.7 build 15160138 (6.7 P01).
Wrapping up
This concludes the 3.9.0 to 3.9.1 VMware Cloud Foundation LCM updates, I hope this post was informative and you find it useful
2 Trackbacks