VMs are shown with absolute path to vmx file in vCenter

These days I had to troubleshoot a problem in a vCenter environment. A few VMs shown with absolute path to vmx file in vCenter inventory. Normally this occurs when a ESXi boots with no connection to its previous mounted storage volumes. VMs located on these volumes are shown like this in inventory.

But this wasn’t the case here: VMs were running and no host was rebooted. Also no storage connections issues were seen. In this post I write about the troubleshooting steps to solve this issue. These steps can also be useful for similar problems.

State at the beginning

According to vCenter – version 6.7 – these VMs were not running. And because of the status of these VMs, it was not possible to do any tasks with vCenter GUI. Most of the context-menu was grayed-out. So Troubleshooting was moved to command-line.

Troubleshooting

Searching for the cause of the issue was leading to a snapshot event. For all these VMs a triggered, failed snapshot changed the name, shown in vCenter.

After finding the cause of the problem, it had to be solved. My first try was to reload VM configuration (vmx file) on the host. This can be done at ESXi console by using vim-cmd:

Reload VM refer to Vmid from previous command:
vim-cmd vmsvc/reload 1

What happened in my case: VM was not listed any more when running vim-cmd vmsvc/getallvms. But no changes in vCenter. But if this works for you, see here how to reload more VMs at once.

Next I wanted to take a look into vmx file. To do so I executed the less command for the file in ESXi console. I connected to the host, showed in vCenter the VM was running on. But I got a file busy error. This occurs normally when you try to run such a command on a host that is NOT running this VM. So find out which host is locking the file. One way is to run:

vmfsfilelockinfo -p Path_to_file

Output looks like this:

The host that locks the file should be shown at “Host owning the lock of file is“. Sometimes this is not working. If so, at least the MAC-address of the lock owning host should be shown. To quickly find the host that owns this MAC, you can use PowerCLI and run:

Get-VMHost | ForEach-Object {$HostName = $_.Name; Write-Output $_;} | Get-VMHostNetworkAdapter | Select-Object @{N="Hostname";E={$HostName}}, DeviceName, Mac, IP

So, locking host was found. I found out this host is not just locking VM’s vmx file, the host is also running the whole VM! To list all running VMs, command esxcli can be used:

esxcli vm process list

Not just the name and the power state of these VMs were wrong in vCenter, even the host was misleading. Luckily my first attempt did solve the problem.

Solution

Because the hosting ESXi was different to the host vCenter showed, I tried to move a affected VM back to the right host. This can be done by PowerCLI again:

Get-VMHost wrong_host | Get-VM *VM_name* | Move-VM -Destination (Get-VMHost right_host)

Where:

• wrong_host: ESXi host, vCenter showed.
• *VM_name*: name of the VM; *’s is important, because the name that is queried includes the absolute path to the vmx file.
• right_host: ESXi host that is actually hosting the VM.

Unsurprisingly this task failed:

[Move-VM Server task failed: The attempted operation cannot be performed in the current state (Powered on).]

But after a few seconds, VM was shown in vCenter by its correct inventory name.