Recovery Locations

Recovery Locations, the host running the recovery service and processing data restores, can be added, edited, and deleted in the Recovery Locations page of the Management Console. They can be added prior to adding devices to the Standby Image plan, or on the first step of the Enable Standby Image plan wizard.

The Recovery Locations page can be found on the Management Console under the Recovery Dashboard section:

The page displays several pieces of information relating to previously created recovery locations.

The columns displayed are:

  • Recovery location name
  • Customer
  • Host availability
    • Offline
    • Online
    • Requires a storage drive

      Drive is required as this is the location the new Virtual Machine files will be created

  • Storage Drive

    The drive is added by clicking Add storage drive, entering the drive letter in the box and clicking save

  • Assigned devices
  • Host storage
  • Host memory capacity
  • Host CPU
  • Host OS
  • Version

Minimum Requirements

  • Windows Admin Center installed on the management machine, available here.
  • Default Hardware Configuration

    By default, each Recovery Location is configured to run 5 restores in parallel with a target VM size of 4 CPU cores and 4 GB of RAM. The following minimal configuration is recommended depending on the restore target:

    • Hyper-V
      • CPU - 22 Cores or more
      • RAM - 32 GB or more
      • Restore Target Drive Capacity - more than two times the size of the selected size of all devices combined
    • Local VHD
      • CPU - 12 Cores or more
      • RAM - 16 GB or more
      • Restore Target Drive Capacity - more than two times the size of the selected size of all devices combined

    Single device restore

    To start Recovery Service on a Windows Server and run a single device restore we recommend the following hardware configuration, depending on the restore format:

    • Hyper-V
      • CPU - 6 Cores or more
      • RAM - 6 GB or more
      • Restore Target Drive Capacity - more than two times the size of the backed up data
    • Local VHD
      • CPU - 4 Cores or more
      • RAM - 5 GB or more
      • Restore Target Drive Capacity - more than two times the size of the backed up data

    Also, make sure that Recovery Location operating system conforms to the requirements stated here: Virtual Disaster Recovery Requirements

    Additional resources for more devices

    The default number of parallel restores can be adjusted depending on the hardware you use for the Standby Image Recovery Location. When configuring each recovery location, it is important to do this in a way it is neither too big (as it might slow down the restore because the host will be overloaded) nor too low (as in this case you might not use the full capacity of your computing resources and hence receive a slower than ideal performance).

    While network bandwidth and disk IOPS don't have any strict requirements to run the Standby Image Recovery Service it does directly affect the restore speed and low resources can cause poor performance.

    CPU and RAM might be a blocker to adding more restores in parallel. If you do not have enough CPU and/or RAM it is possible to see performance degradation, failing restores, or Virtual Machine boot issues due to “out of memory“ errors.

    We recommend reserving the following additional resources for each extra device when configuring a number of parallel restores:

    • Hyper-V
      • CPU - 4 cores
      • RAM - 4 GB
    • Local VHD
      • CPU - 2 cores
      • RAM - 2 GB

    The requirements mentioned above are recommendations. You may set the number of cores and memory at your discretion, however, taking into account the recommendations above.

    If you have already had experience using Recovery Console and have a suitable configuration, it is possible to use the same configuration for Standby Image when taking into account the recovery of the same number of devices of a similar configuration.

    Recommendations for Maximum Performance

    Hardware

    When running multiple restores in parallel, we recommend the following to increase the performance of each machine:

    • Use SSD disks with higher IOPs
    • A fast network connection with good bandwidth

    Both the target disk and system disk should have enough performance as the system disk may be used by system services and Hyper-V.

    Hypervisor Configuration

    While required to effectively mitigate certain classes of vulnerabilities, the core scheduler (enabled by default for Windows Server 2019 and newer) may also potentially reduce performance. We recommend changing the scheduler to Classic to increase performance.

    This action should be done along with applying appropriate security controls to mitigate risks raised by this change.

    The free Hyper-V Server 2019 ISO can be found here.

    Anti-Virus Recommended Exclusions

    The following are recommended to add to the anti-virus exclusions list to allow for successful backups:

    Recovery Service Exclusions

    • AuthTool.exe [file] SYSTEM_DRIVE:\Program Files\Recovery Service\*\AuthTool.exe
    • unified_entry.exe [file]. SYSTEM_DRIVE:\Program Files\Recovery Service\*\unified_entry.exe
    • RecoveryFP.exe [file] SYSTEM_DRIVE:\Program Files\Recovery Service\*\BM\RecoveryFP.exe
    • VdrAgent.exe [file] SYSTEM_DRIVE:\Program Files\Recovery Service\*\BM\VdrAgent.exe
    • ProcessController.exe [file] SYSTEM_DRIVE:\Program Files\Recovery Service\*\BM\ProcessController.exe
    • ClientTool.exe [file] SYSTEM_DRIVE:\Program Files\Recovery Service\*\BM\ClientTool.exe

    Virtual Machines Exclusions

    • *.vhd RESTORE_TARGET_DRIVE\StandbyImage\*\vm\*.vhd
    • *.vhdx RESTORE_TARGET_DRIVE\StandbyImage\*\vm\*.vhdx

    Hyper-V Processes Exclusions

    • %ProgramData%\Microsoft\Windows\Hyper-V [folder]
    • Vmms [process]. %systemroot%\System32\Vmms.exe
    • Vmwp [process]. %systemroot%\System32\Vmwp.exe
    • Vmsp [process]. %systemroot%\System32\Vmsp.exe
    • Vmcompute [process]. %systemroot%\System32\Vmcompute.exe

    System Network Configuration

    When running a large number of parallel restores on the same recovery location, (around 50) a lot of network connections may be utilized. In order to improve the performance and stability of such heavily loaded systems it is recommend to adjust network configuration in the following way:

    1. Increase ephemeral ports count, set up new values:
      1. start = 20000
      2. number of ports = 45000
    2. Reduce TcpTimedWaitDelay to 5 seconds
    3. See details here: Settings that can be Modified to Improve Network Performance - BizTalk Server