Recovery Locations

Recovery Locations, the host running the recovery service and processing data restores, can be added, edited, and deleted in the Recovery Locations page of the Management Console. They can be added prior to adding devices to the Standby Image plan, or on the first step of the Enable Standby Image to Hyper-V, Enable Standby Image to Azure and Enable Standby Image to ESXi wizards.

The Recovery Locations page can be found on the Management Console under the Continuity section:

The page displays several pieces of information relating to previously created recovery locations.

The columns displayed are:

• Recovery location name
• Customer
• Recovery location type
  • Azure
  • Hyper-V
  • VMware ESXi
• Host availability
  • Offline
  • Online
  • Requires a storage drive

    Drive is required for Standby Image and Recovery Testing locations as this is the location in the file system where the new Virtual Machine files will be created. This is not required for Azure and ESXi locations

• Storage location

  For Hyper-V, this can be added by clicking Add storage location, entering the drive letter or local path in the box and clicking Save

• Assigned devices
• Host storage
• Host memory capacity
• Host CPU
• Host OS
• Version

Minimum Requirements

For Standby Image and Recovery Testing, Windows Admin Center must be installed on the management machine, available here.

This is not required for One-Time Restore.

Default Hardware Configuration

By default, each Recovery Location is configured to run 5 restores in parallel with a target VM size of 4 CPU cores and 4 GB of RAM. The following minimal configuration is recommended depending on the restore target:

• Hyper-V and ESXi
  • CPU - 22 Cores or more
  • RAM - 32 GB or more
  • Restore Target Drive Capacity - more than two times the size of the selected size of all devices combined
• Local VHD and Azure
  • CPU - 12 Cores or more
  • RAM - 16 GB or more
  • Restore Target Drive Capacity - more than two times the size of the selected size of all devices combined

Single device restore

To start Recovery Service on a Windows Server and run a single device restore we recommend the following hardware configuration, depending on the restore format:

• Hyper-V and ESXi
  • CPU - 6 Cores or more
  • RAM - 6 GB or more
  • Restore Target Drive Capacity - more than two times the size of the backed up data
• Local VHD and Azure
  • CPU - 4 Cores or more
  • RAM - 5 GB or more
  • Restore Target Drive Capacity - more than two times the size of the backed up data

Also, make sure that Recovery Location operating system conforms to the requirements stated here: Virtual Disaster Recovery Requirements

Azure Requirements

To install the Recovery Location's recovery service on an Azure VM, the following requirements are necessary:

• A user created for you in your Azure tenant. The user must:
  • Have access to a subscription
  • Have access to a resource group you want to use to keep the Recovery Location VM and restored (target) Azure VMs
  • Be able to assign permissions on virtual machines within the resource group

Additional resources for more devices

The default number of parallel restores can be adjusted depending on the hardware you use for the Standby Image Recovery Location. When configuring each recovery location, it is important to do this in a way it is neither too big (as it might slow down the restore because the host will be overloaded) nor too low (as in this case you might not use the full capacity of your computing resources and hence receive a slower than ideal performance).

While network bandwidth and disk IOPS don't have any strict requirements to run the Standby Image Recovery Service it does directly affect the restore speed and low resources can cause poor performance.

CPU and RAM might be a blocker to adding more restores in parallel. If you do not have enough CPU and/or RAM it is possible to see performance degradation, failing restores, or Virtual Machine boot issues due to “out of memory“ errors.

We recommend reserving the following additional resources for each extra device when configuring a number of parallel restores:

• Hyper-V and ESXi
  • CPU - 4 cores
  • RAM - 4 GB
• Local VHD and Azure
  • CPU - 2 cores
  • RAM - 2 GB

Recommendations for Maximum Performance

Hardware

When running multiple restores in parallel, we recommend the following to increase the performance of each machine:

• Use SSD disks with higher IOPs
• A fast network connection with good bandwidth

Both the target disk and system disk should have enough performance as the system disk may be used by system services and Hyper-V.

Hypervisor Configuration

While required to effectively mitigate certain classes of vulnerabilities, the core scheduler (enabled by default for Windows Server 2019 and newer) may also potentially reduce performance. We recommend changing the scheduler to Classic to increase performance.

This action should be done along with applying appropriate security controls to mitigate risks raised by this change.

The free Hyper-V Server 2019 ISO can be found here.

Anti-Virus Recommended Exclusions

The following are recommended to add to the anti-virus exclusions list to allow for successful backups:

Recovery Service Exclusions

• AuthTool.exe [file] SYSTEM_DRIVE:\Program Files\Recovery Service\*\AuthTool.exe
• unified_entry.exe [file]. SYSTEM_DRIVE:\Program Files\Recovery Service\*\unified_entry.exe
• RecoveryFP.exe [file] SYSTEM_DRIVE:\Program Files\Recovery Service\*\BM\RecoveryFP.exe
• VdrAgent.exe [file] SYSTEM_DRIVE:\Program Files\Recovery Service\*\BM\VdrAgent.exe
• ProcessController.exe [file] SYSTEM_DRIVE:\Program Files\Recovery Service\*\BM\ProcessController.exe
• ClientTool.exe [file] SYSTEM_DRIVE:\Program Files\Recovery Service\*\BM\ClientTool.exe

Virtual Machines Exclusions

• *.vhd RESTORE_TARGET_DRIVE\StandbyImage\*\vm\*.vhd
• *.vhdx RESTORE_TARGET_DRIVE\StandbyImage\*\vm\*.vhdx

Hyper-V Processes Exclusions

• %ProgramData%\Microsoft\Windows\Hyper-V [folder]
• Vmms [process]. %systemroot%\System32\Vmms.exe
• Vmwp [process]. %systemroot%\System32\Vmwp.exe
• Vmsp [process]. %systemroot%\System32\Vmsp.exe
• Vmcompute [process]. %systemroot%\System32\Vmcompute.exe

System Network Configuration

When running a large number of parallel restores on the same recovery location, (around 50) a lot of network connections may be utilized. In order to improve the performance and stability of such heavily loaded systems it is recommend to adjust network configuration in the following way:

1. Increase ephemeral ports count, set up new values:
   1. start = 20000
   2. number of ports = 45000
2. Reduce TcpTimedWaitDelay to 5 seconds

See details here: Settings that can be Modified to Improve Network Performance - BizTalk Server