VMware Shutdown - vSAN Cluster

In the following examples, a VMware vSAN Cluster is protected by a Single or Advanced UPS Configuration. vCenter Server is running on a Virtual Machine.

Recommended Deployment

PowerChute can be located on a physical Windows machine outside the vSAN Cluster or deployed as a VM inside the vSAN Cluster. The vCenter Server account configured in PowerChute Network Shutdown must have Administrator permissions on vCenter Server and on each of the ESXi hosts being managed by PowerChute. This can be an Active Directory account or a local user account. For more information see Active Directory VMware Configuration.

Example 1: Single UPS, 2-Node Stretch Cluster with Witness Appliance and Management Host

Setup

• PowerChute is installed on a physical Windows machine.

• Single UPS configuration.

• Delay Maintenance Mode is enabled (required for vSAN hosts) with duration set to X seconds.

• vCenter Server is running on a VM inside the vSAN Cluster.

• vSAN Witness Appliance is deployed on the Management Host and added to the inventory as a Host.

• VM Prioritization is enabled.

• An Active Directory controller VM is added to the High priority group with appropriate duration for shutdown and startup. This VM needs to be shut down after the vCenter Server VM and started before it.

• vCenter Server VM is added to the Medium priority group with appropriate duration for shutdown and startup.

• The option to turn off the UPS or Outlet Group is enabled.

• A shutdown command file has been configured with a 120 second duration.

When a critical UPS event, such as UPS on Battery occurs the following sequence is triggered:

Shutdown Sequence

1. PowerChute reports that the UPS is on battery.

2. Shutdown delay for the On Battery event elapses. PowerChute sends a command to turn off the UPS or Outlet Group.

3.  PowerChute starts VM and vApp shutdown on each Host.

4. PowerChute gracefully shuts down the vCenter Server VM in the Medium priority group followed by the Active Directory controller VM in the High priority group during VM Shutdown.

5. VM/vApp Shutdown durations elapse.

6. PowerChute starts executing the shutdown command file.

7. Shutdown command file duration elapses and PowerChute starts a Maintenance Mode task on the first vSAN host, and waits the Delay Host Maintenance Mode Duration. If there is a vSAN synchronization active, PowerChute will wait the specified vSAN Synchronization Duration and check if it has completed with respect to the value set for "vsan_synch_retry_time" in the pcnsconfig.ini file until data re-synchronization is no longer active, or the retry limit has been reached. PowerChute then shuts down the host.

8. Once the Delay Host Maintenance Mode Duration has elapsed, PowerChute will start a Maintenance Mode task on the next vSAN host, and waits the Delay Host Maintenance Mode Duration. If there is a vSAN synchronization active, PowerChute will wait the specified vSAN Synchronization Duration and check if it has completed with respect to the value set for "vsan_synch_retry_time" in the pcnsconfig.ini file until data re-synchronization is no longer active, or the retry limit has been reached. PowerChute then shuts down the host.

9. PowerChute starts a Maintenance Mode task on the Witness Host, waits the Delay Host Maintenance Mode Duration and shuts down the host.

10. PowerChute starts a Maintenance Mode task on the Management Host, waits the Delay Host Maintenance Mode Duration of X seconds  and shuts down the host.

11. OS shutdown sequence starts on the PowerChute physical machine. After a 70 second delay, the OS starts to shut down.

12. UPS waits for the duration that is greatest of Low Battery Duration/Maximum Required Delay (Non-Outlet Aware UPS's) or the Outlet Group Power Off Delay.

13. After this delay, a further non-configurable two-minute delay is counted down.

14. UPS turns off after the user-configurable Shutdown Delay time has elapsed or the Outlet Group turns off after the power off Delay elapses.

NOTE: In a vSAN configuration, Witness and Management hosts will get placed into Maintenance Mode and shut down after Cluster hosts if Delay Maintenance Mode is enabled. vSAN Hosts are placed into Maintenance mode using “No data migration” for the vSAN data evacuation mode.

Example 2: Advanced UPS Configuration, vSAN Stretch Cluster with Witness Appliance and Management Host

NOTE: PowerChute could be installed on a VM on the Management Host.

Setup

• Advanced configuration containing 4 single UPS setups:

• • UPS Setup 1: PowerChute is installed on a physical Windows machine.

  • UPS Setup 2: Primary site ESXi Host A, ESXi Host B, ESXi Host C

  • UPS Setup 3: Secondary site ESXi Host A, ESXi Host B, ESXi Host C

  • UPS Setup 4: Witness Host and Management Host

• Delay Maintenance Mode is enabled (required for vSAN hosts) with duration set to X seconds.

• Shut down all Cluster VMs is enabled.

• vCenter Server is running on a VM inside the vSAN Cluster on the primary site.

• vCenter Server VM is added to the High priority group with appropriate duration for shutdown and startup.

• The option to turn off the UPS or Outlet Group is enabled.

• A shutdown command file has been configured with a 120 second duration.

When a critical UPS event, such as UPS on Battery occurs on the primary site, the following sequence is triggered:

Shutdown Sequence

1. PowerChute reports that the UPS protecting the primary site (vSAN Cluster) is On Battery.

2. Shutdown delay for the On Battery event elapses. PowerChute sends a command to turn off the UPS or Outlet Group.

3. PowerChute determines that Fault Tolerance Threshold (FTT) has not been exceeded, since the number of critical groups = 1, which is not greater than the FTT Level of 1. PowerChute starts Virtualization shutdown tasks on the primary site only.

4. PowerChute migrates the vCenter Server VM in the High priority group during VM Migration. Other VMs are also migrated to Hosts on the secondary site during this step.

5. Any VMs or vApps that could not be migrated are shut down.

6. VM/vApp Shutdown durations elapse. PowerChute starts executing the shutdown command file.

7. Shutdown command file duration elapses and PowerChute starts a Maintenance Mode task on the first vSAN host on the primary site with "Ensure Data Accessibility" as FTT is not exceeded. The Delay Host Maintenance Mode Duration is waited and if there is a vSAN synchronization active, PowerChute will wait the specified vSAN Synchronization Duration and check if it has completed with respect to the value set for "vsan_synch_retry_time" in the pcnsconfig.ini file until data re-synchronization is no longer active, or the retry limit has been reached. PowerChute then shuts down the host.

8. PowerChute starts a Maintenance Mode task on the next vSAN host on the primary site with "Ensure Data Accessibility". The Delay Maintenance Mode Duration is waited and if there is a vSAN synchronization active, PowerChute will wait the specified vSAN Synchronization Duration and check if it has completed with respect to the value set for "vsan_synch_retry_time" in the pcnsconfig.ini file until data re-synchronization is no longer active, or the retry limit has been reached. PowerChute then shuts down the host.

9. PowerChute starts a Maintenance Mode task on the next vSAN host on the primary site with "Ensure Data Accessibility". The Delay Maintenance Mode Duration is waited and if there is a vSAN synchronization active, PowerChute will wait the specified vSAN Synchronization Duration and check if it has completed with respect to the value set for "vsan_synch_retry_time" in the pcnsconfig.ini file until data re-synchronization is no longer active, or the retry limit has been reached. PowerChute then shuts down the host.

10. UPS waits for the duration that is greatest of Low Battery Duration/Maximum Required Delay (Non-Outlet Aware UPS's) or the Outlet Group Power Off Delay.

11. After this delay, a further non-configurable two-minute delay is counted down.

12. UPS turns off after the user-configurable Shutdown Delay time has elapsed or the Outlet Group turns off after the power off Delay elapses.

NOTE: In a vSAN configuration, Witness and Management hosts will get placed into Maintenance Mode and shut down after Cluster hosts if Delay Maintenance Mode is enabled.

NOTE: If Fault Tolerance Threshold (FTT) is enabled, vSAN hosts are placed into Maintenance Mode using "Ensure Data Accessibility" if the number of critical groups is less than or equal to the FTT Level.

Putting a host into Maintenance Mode with "Ensure Data Accessibility" can trigger data re-synchronization on the host. In this event, PowerChute will wait until the data re-synchronization is complete (with retry limit) before placing the host into maintenance mode and shutting it down. See Host Maintenance Mode for more information.

Example 3: Advanced UPS Configuration, 3 Node vSAN Standard Cluster

Setup

• Advanced configuration containing 4 single UPS setups:

• • UPS Setup 1: PowerChute is installed on a physical Windows machine.

  • UPS Setup 2: ESXi Host A

  • UPS Setup 3: ESXi Host B

  • UPS Setup 4: ESXi Host C

• Delay Maintenance Mode is enabled (required for vSAN hosts) with duration set to X seconds.

• Fault Tolerance Threshold is enabled, FTT Level is set to 1.

• Shut down all Cluster VMs is enabled.

• vCenter Server is running on a VM inside the vSAN Cluster.

• vCenter Server VM is added to the High priority group with appropriate duration for shutdown and startup.

• The option to turn off the UPS or Outlet Group is enabled.

• A shutdown command file has been configured with a 120 second duration.

Critical Event 1

When a critical UPS event, such as UPS on Battery occurs on a host in the vSAN Cluster, for example, ESXi Host C, the following sequence is triggered:

Shutdown Sequence

1. PowerChute reports that the UPS Setup 3 (ESXi Host C) is On Battery .

2. Shutdown delay for the On Battery event elapses. PowerChute sends a command to turn off the UPS or Outlet Group.

3. PowerChute determines that Fault Tolerance Threshold has not been exceeded, since the number of critical groups = 1 which is not greater than the FTT Level of 1. PowerChute starts Virtualization Shutdown tasks on ESXi Host C . PowerChute migrates VMs to non-critical Hosts (ESXi Host A, ESXi Host B) in the vSAN Cluster

4. VM Migration durations elapse. PowerChute starts VM/vApp Shutdown.

5. VM/vApp Shutdown durations elapse. PowerChute starts executing the shutdown command file.

6. Shutdown command file duration elapses and PowerChute starts a Maintenance Mode task on ESXi Host C with "Ensure Data Accessibility" as FTT Level is not exceeded. The Delay Host Maintenance Mode Duration is waited and if there is a vSAN synchronization active, PowerChute will wait the specified vSAN Synchronization Duration and check if it has completed with respect to the value set for "vsan_synch_retry_time" in the pcnsconfig.ini file until data re-synchronization is no longer active, or the retry limit has been reached. PowerChute then shuts down the host. 

7. UPS waits for the duration that is greatest of Low Battery Duration/Maximum Required Delay (Non-Outlet Aware UPS's) or the Outlet Group Power Off Delay.

8. After this delay, a further non-configurable two-minute delay is counted down.

9. UPS turns off after the user-configurable Shutdown Delay time has elapsed or the Outlet Group turns off after the power off Delay elapses.

NOTE: If Fault Tolerance Threshold (FTT) is enabled, vSAN hosts are placed into Maintenance Mode using "Ensure Data Accessibility" if the number of critical groups is less than or equal to the FTT Level.

Putting a host into Maintenance Mode with "Ensure Data Accessibility" can trigger data re-synchronization on the host. In this event, PowerChute will wait until the data re-synchronization is complete (with retry limit) before placing the host into maintenance mode and shutting it down. See Host Maintenance Mode for more information.

Critical Event 2

When a critical UPS event, such as UPS on Battery occurs on a host in the vSAN Cluster, for example, ESXi Host B, the following sequence is triggered:

NOTE: ESXi Host C is already critical.

Shutdown Sequence

1. PowerChute reports that the UPS Setup 2 (ESXi Host B) is On Battery .

2. Shutdown delay for the On Battery event elapses. PowerChute sends a command to turn off the UPS or Outlet Group.

3. PowerChute determines that Fault Tolerance Threshold has been exceeded since the number of critical groups = 2 which is greater than the FTT Level of 1.
   PowerChute starts Virtualization Shutdown on critical host ESXi Host B and non-critical host ESXi Host A as Shut down all Cluster VMs is enabled.

4. VM Migration durations elapse. PowerChute starts VM/vApp Shutdown.

5. PowerChute gracefully shuts down the vCenter Server VM in the High priority group during VM Shutdown of ESXi Host A.

6. VM/vApp Shutdown durations elapse. PowerChute starts executing the shutdown command file. 

7. Shutdown command file duration elapses and PowerChute starts a Maintenance Mode task on ESXi Host B with "No Data Migration" action as FTT Level has been exceeded. The Delay Maintenance Mode Duration is waited and if there is a vSAN synchronization active, PowerChute will wait the specified vSAN Synchronization Duration and check if it has completed with respect to the value set for "vsan_synch_retry_time" in the pcnsconfig.ini file until data re-synchronization is no longer active, or the retry limit has been reached. PowerChute then shuts down the host.

8. UPS waits for the duration that is greatest of Low Battery Duration/Maximum Required Delay (Non-Outlet Aware UPS's) or the Outlet Group Power Off Delay.

9. After this delay, a further non-configurable two-minute delay is counted down.

10. UPS turns off after the user-configurable Shutdown Delay time has elapsed or the Outlet Group turns off after the power off Delay elapses.

Copyright © 2020 Schneider Electric
All rights reserved
http://www.apc.com
Last Updated: 17 December, 2020
990-4595H
 
 Send any comments on this help to:
IE-PowerChute-feedback@schneider-electric.com