I love the vCenter Server Appliance. The migration works pretty well. Still from time to time I stumble across minor problems (which until now were always quite easy to workaround/fix).
One of this migration ‘issues’ I was faced with recently at a customers site.
We migrated a vCenter against an ESXi host which was using a distributed switch and the corresponding portgroup as a target network.
Since we add the virtual network adapter directly on the ESXi host to the distributed switch we need to have an ephemeral portgroup (otherwise only the vCenter could add the VMs network adapter to this portgroup).
The general process of the migration look like the following.
- Deploy a new and empty vCenter Server appliance and connect it to the network
- A temporary IP-address is given to this vCenter Server appliance
- All relevant data of the source windows based vCenter Server is exported and transferrred over the network to the new vCenter Server appliance
- When the whole data-set is transferred, shutdown the original vCenter and give the new vCSA the network identity of the original vCenter
Unfortunately the last step was not working properly. After a certain amount of time (and coffee) the migration process has stuck at 50% – Shutting down source machine
Since we had other stuff to do we repeated the steps and did the migration wizard again……..
……… with the same result (after even more coffee).
Now it was time to analyze it further. I recognized that neither the temporary nor the static vCenter IP was reachable on the network.
Connecting to the ESXi host directly looking at the network configuration offered the following:
The port status of the new vCenter VM was blocked. Digging through further logs we figured out that the vCenter Server appliance tried to change the mac address.
That is a setting that is by default not allowed within the security policy of a distributed switch portgroup.
The good thing is: During the migration the original VM remains ‘untouched’, therefore a fallback is quite easy. Reboot the original windows based vCenter VM and wait until all services are up again.
Important: Change the security policy within the network settings of your distributed switch portgroup where the vCenter Server appliance will be connected to.
Maybe a feature that can be included in the migration-pre checking if you ask me. Maybe there are also other security mechanisms in your network messing up with a successful migration here!
Maybe @Adam Eckerle is reading this and can give forward this information :)
Hey, just wanted to thank you since I had the exact same issue and that fixed it for me :)
Hello Fabian,
we have the same problem but change of the security policy doesn’t help. The temporary network was lost, on the new VCSA console I see the message “Upgrade EXPORT failed”.
Do you have an idea ?
best regards
Dirk Ehlers
Have you tried to ping the new vCSA during the process? Can you ping it with the temporary IP? Can you afterwards ping it with the static IP?
Is the vCenter connected to a distributed oder standard vSwitch? If distributed, make sure it is ephemeral.
I ping the new vCSA on temporary IP from begin of Migration State 2 until stuck at 50%. I don’t try to ping the static IP because I see on the new vCSA Console that it still has the temporary IP.
The vCenter is connected to a distributed vSwitch but it is static. On this distributed switch are connected about 50 VMs. If I change these settings from static to ephemeral, is the network traffic interrupted ?
Create a new port group that is ephemeral in the same vlan.
Change the security setting and add the new vcsa to this portgroup
I have tried this, same result. Ping lost on new vCSA (temp / static IP). I logged with SSH shell on the ESX Host where the new vCSA is and try to ping, no answer. Are there any migration logs written on the new vCSA during the migration process ?
I found the error:
After last failed migration I disabled in the old vCenter the network connect from the new vCenter. I started the new vCenter and opened a web console. With “Alt+F1” I could open a separate console and start “bash”. I changed to directory /var/log/vmware/upgrade. Then I looked in the file “upgrade-export.log”.
The old vCenter was configured with a DHCP reservation. I see in the upgrade-export.log that the migration process change the network adapter from temporary static IP address to DHCP address, but the network adapter get no answer from a DHCP Server. It restores the old temporary IP address and wrotes in the upgrade-export.log “Network FAILED to restart using new parameters”.
I don’t use the new port group with ephemeral, I tried the static one, it works.
Many thanks for your help, Fabian :)
When upgrading the VCSA from 6.5 to 6.7 I had a similar problem. For me, the issue was that the old VCSA wasn’t completely shut down before the upgrade wizard tried to switch its IP to the new VCSA. The fix in my case was to disconnect the network connection from the old VCSA as soon as its shut down sequence started. The new VCSA took over the IP properly once I did that. I connected to the old VCSA via the VMware Remote Console launched via the ESXi host’s own web console. I used the Remote Console to disconnect the NIC on the old VCSA.
I have solved this problem with this steps:
1. Uninstalled Vmware Update Manager
2. Mount VCSA ISO from datastore (not from network and etc..)