PSP rule for active/standby controller arrays (like Nimble Storage)

First, what does active/standby mean. For this blog post, it means the array hast at least 2 controllers, one presents ALL LUNs to the hosts, the second controller presents NO active LUNs path to the hosts. Just in case of a manual or automatic failover, the second controller takes-over ALL LUN-presentations – and active paths. Examples of such arrays are:

• [very old one] HP MSA 1000
• HPE Nimble Storage
• Dell EMC SC Series Compellent

VMware changed a default setting in ESXi 6.7 that controls the handling of broken storage paths. This Setting is action_OnRetryErrors and is part of Storage Array Type Plug-In (SATP). In 6.0 and 6.5 default was OFF, since 6.7 default if ON. Put simply: this setting turned on can lead to no device access when controller fails over! More information about the behavior of action_OnRetryErrorsyou can found here and here.

When already on 6.7, check running setting using command beneath. When ON, create a rule, to turn it OFF. When a rule already exists without setting action_OnRetryErrors, delete it and create it like beneath.

Here I will show commands for a Nimble array as example. To find necessary information for your storage see my post here. For a more detailed example of creating PSP rule see my post for 3PAR (an active/active array) here.

In ESXi shell

To show current settings on device, run:

esxcli storage nmp device list

In line Storage Array Type Device Config you see action_OnRetryErrors=off or on

Check if an rule exists for the array vendor

esxcli storage nmp satp rule list | grep -i nimble

Run this command in ESXi (console or SSH) to create rule

esxcli storage nmp satp rule add --psp=VMW_PSP_RR --satp=VMW_SATP_ALUA --psp-option='policy=iops;iops=1' --option disable_action_OnRetryErrors --vendor=Nimble --description='HPE Nimble Rule'

Using PowerShell/PowerCLI

Before you can run PowerCLI commands, you need to establish a connection to a vCenter or host by running a Connect-VIServer command. For all following PowerCLI commands, a ESXCLI-object must be created. To do so, run:

$esxcli = Get-VMHost hostname/IP-address | Get-EsxCli -V2

To show current settings on device, run:

$esxcli.storage.nmp.device.list.Invoke() | select device, StorageArrayTypeDeviceConfig

You can also see action_OnRetryErrors=off or on.

Run this commands to create rule:

$esxcli.storage.nmp.satp.rule.add.Invoke(@{"satp"="VMW_SATP_ALUA"; "vendor"="Nimble"; "psp"="VMW_PSP_RR"; "description"="HPE Nimble Rule"; "pspoption"="policy=iops;iops=1"; "option"="disable_action_OnRetryErrors"})

Check if an rule exists for the array vendor:

$esxcli.storage.nmp.satp.rule.list.Invoke() | where {$_.vendor -eq "Nimble"}

 Notes

• Vendor and Model is case sensitive!
• The simplest way to apply and test new rule is to reboot hosts.
• get-esxcli can only run against one host.