vCenter does not start after changing DNS servers

Recently I had to change DNS server in a VMware vSphere vCenter Appliance (VCSA). Normally this works without further issues. This time it was different. vCenter services did not start after changing DNS servers. I describe in this short post what went wrong and how I could fixed it.

I am aware of two ways to change DNS server in VCSA. First is by using the vCenter Server Appliance Management Interface (VAMI) on port 5480. In this web console you can edit network settings to update DNS servers.

Unfortunately this do not work very well often. Therefor I switched to the second method. Here, DNS can be changed in console of VCSA VM.

Task is straightforward and feels like on a ESXi host. To complete the task I normally reboot the VM. When I did this this time, I realized that VMware services did not start. To check this, command

service-control --all --status

can be used. You will see running, stopped and pending services. Most of the services were still stopped many minutes after the reboot. After investigating some logs, I found out there seemed to be a problem by connecting to ports addressed by hostname localhost. Like in file vmon-syslog.log:

[...health xml from url http ://localhost:16666/cls/cmhealthstatus. Exception: <urlopen error [Errno 111] Connection refused>]

Pinging localhost showed a unexpected message:

[ping: unknown host localhost]

So, name resolution for localhost did not work. After a look into /etc/hosts everything was clear. There was no entry for localhost. Just added a line at the end of the file – right after # VAMI_EDIT_END:

127.0.0.1 VCSA_FQDN VCSA_shortname localhost

Please replace VCSA_FQDN and VCSA_shortname accordingly.

With next reboot, all services started without any issue. Reboot was necessary, just starting services manually did not work.

If anyone has a better solution, let me know.

Notes

• vCenter version in my case: 6.7.0.50000.
• This vCenter was installed by using IP-address instead of FQDN. I can imagine that this leads to this erroneous behavior.
• Because this vCenter will be upgraded to 7.0 very soon, I did no deep research for a clean solution.
• Manually added entry survived reboots.
• If you have problems by changing proxy server in VCSA, this post maybe helps.