In this article I’m going to demonstrate the steps required to perform an incremental online LCM update from VMware Cloud Foundation 4.0 to 4.0.1, specifically on the management workload domain.
VCF 4.0.1 released 25/06/2020 and this is release notes link
Note that I used the word incremental to identify that LCM is going to apply each and every bundle in between to get to the target version, without skipping in-between versions. This is in contrast with the skip-level upgrade introduced with 3.10 and 4.0+, for which I’m probably going to write a separate article.
The components that are going to be upgraded are:
- SDDC Prechecks: repeat this step before each bundle is applied
- SDDC Manager 4.0.0.1 (bundle ID 6b5d5a0e-5bdc-4679-82c3-860bd660ea83)
- from 4.0.0.0 build 16008466 to 4.0.0.1 build 16221345
- vCenter Server 7.0.0a (bundle ID fb468d09-bdb4-4834-a762-953a4d688b23)
- from 7.0.0.10100 build 15952498 to 7.0.0.10300 build 16189094
- SDDC Manager 4.0.1.0 (bundle ID a188cedb-e56f-482d-8a13-7daa36ff7d46)
- from 4.0.0.1 build 16221345 to 4.0.1.0 build 16428904
- SDDC Manager 4.0.1.0 configuration drift (bundle ID f1465cdc-c72d-49f2-976c-f68eca41f1e6)
- same build is kept 4.0.1.0 16428904. This bundle is doing clean-up and under the hood fixes
- NSX-T 3.0.1 (bundle ID 29753b80-2bf4-4604-85c3-86f49d610153)
- from 3.0.0.0.0 build 15946738 to 3.0.1.0.0 build 16404613
- vCenter Server 7.0.0b (bundle ID bb41c565-2d80-47bb-b4ea-8ed532a003b0)
- from 7.0.0.10300 build 16189094 to 7.0.0.10400 build 16386292
- ESXi 7.0.0b (bundle ID 168ee409-d2b7-40e5-9d77-1dc3cec75e07)
- from 7.0.0 build 15843807 to 7.0.0 build 16324942
1.SDDC System Prechecks
Always perform a precheck before you start, it’s going to highlight any problem that needs fixing. Head over to the management workload domain, tab Update/Patches > PRECHECK button
2.SDDC Manager 4.0.0.1
This bundle is going to update all SDDC Manager services
3.vCenter Server 7.0.0a
Because the PSC is embedded on vCenter, while you perform this upgrade you might lose the SDDC Manager UI since since SDDC Manager authenticates against the built-in PSC. That’s perfectly if does happen you simply wait for vCenter Server to be back online and log back in on SDDC Manager.
4.SDDC Manager 4.0.1.0
This bundle is again, going to update all SDDC Manager services
5.SDDC Manager 4.0.1.0 configuration drift
This is a very quick bundle to apply, in fact it’s merely updating the internal SDDC database, even in my nested lab (supposedly slower than a physical VCF setup) it took less than 2 minutes.
6.NSX-T 3.0.1
Here we’re select the management domain cluster
Note that the NSX-T Manager does not get upgraded on the same run, in my lab I had to re-apply the same NSX-T bundle in order to get the NSX-T Manager upgraded.
7.vCenter Server 7.0.0b
8.ESXi 7.0.0b
Note the cluster level selection feature is now available 🙂
I’m running nested and ESXi failed to update with the following error on UI
Checking on /var/log/vmware/vcf/lcm/lcm.log would display a more comprehensive error message (bold)
[vcf_lcm,b07a11c6520e3ec3,cd71,upgradeId=955c13e7-c024-4daa-8e09-ece469302edc,resourceType=ESX_HOST,resourceId=0c5b8771-6928-4196-9a67-6d4840553f5c,bundleElementId=64e176ac-11c0-4c10-b91e-455ad1002100] [c.v.e.s.l.p.i.vsan.VsanHealthService,ThreadPoolTaskExecutor-10] VSAN Health: VSAN HCL Info: The controller device vmhba0:VMware Inc. PVSCSI SCSI Controller that is running with driver pvscsi:0.1-2vmw.700.1.0.15843807 does not show up on HCL.
Set to false the following three LCM options to false in /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties
vsan.healthcheck.enabled=false
vsan.hcl.update.enabled=false
vsan.precheck.enabled=false
Restart lcm and your’e good to go. However the next issue for was me was that the host did not automatically exit maintenance mode, from lcm.log I could see the following (not so useful) error message.
2020-12-24T09:50:06.633+0000 INFO [vcf_lcm,e1f1fe9882e04d53,d0da,upgradeId=9d650475-dca6-4f2f-bdcc-1762384db355,resourceType=ESX_HOST,resourceId=93dc8b07-5cd3-4c23-9fc5-b504d36d584f,bundleElementId=64e176ac-11c0-4c1oryClient,ThreadPoolTaskExecutor-10] logical inventory - update entity status; type ESXI id 93dc8b07-5cd3-4c23-9fc5-b504d36d584f status ERROR 2020-12-24T09:50:06.635+0000 DEBUG [vcf_lcm,e1f1fe9882e04d53,d0da,upgradeId=9d650475-dca6-4f2f-bdcc-1762384db355,resourceType=ESX_HOST,resourceId=93dc8b07-5cd3-4c23-9fc5-b504d36d584f,bundleElementId=64e176ac-11c0-4c1fiedRestTemplate,ThreadPoolTaskExecutor-10] Writing [InventoryEntityUpdateSpec(type=ESXI, status=ERROR, version=null)] as "application/json" "status": "ERROR"
The VUM baseline was correctly attached temporarily to each node
I couldn’t find what was wrong in my setup so I ended up manually exiting maintenance mode and kick-off the ESXi LCM update bundle again.
If I find out what was causing this problem I will update the article; but for now that’s it! We have successfully completed the 4.0 to 4.0.1 VMware Cloud Foundation Update.
1 Trackback