Use ibackup with warm standby

The warm standby and the backup and restore features require careful planning to use together.

Warm standby and ibackup features of Splunk SOAR (On-premises) use the Write Ahead Logging feature in PostgreSQL. When you configure a Splunk SOAR (On-premises) deployment to use both warm standby and ibackup, you must configure warm standby first. After restoring a deployment with ibackup, you must update the warm standby configuration.

The ibackup feature should only be configured on the primary instance.

Configuring warm standby after configuring ibackup archives all existing backups. Archiving all of the backups prevents new backups from being generated or existing backups from being used in a restore.

You can generate new backups once you run ibackup with the --setup option.

Restore a system configured for warm standby

In a warm standby configuration, when the primary Splunk SOAR (On-premises) instance is restored from a backup, you must update the warm standby configuration.

Prerequisites

You need the following information to update your warm standby configuration:

Password for the Splunk SOAR (On-premises) user on the secondary Splunk SOAR (On-premises) instance. If the Splunk SOAR (On-premises) user does not have a password, you must set one.
Password for the PostgreSQL database replication user.
Configuration information for creating the SSL certificate:
- Country code
- State code
- Organization
- Organization unit
- Domain
- Email

Restore a backup from a warm standby primary to the same Splunk SOAR (On-premises) instance

When you restore a backup of a Splunk SOAR (On-premises) warm standby primary to the same instance, the warm standby configuration must be updated.

To update the warm standby configuration, perform the following steps:

Open a terminal session for both the primary and secondary Splunk SOAR (On-premises) instances. Keep these sessions open until you complete these steps.
1. From the command line, SSH to your primary Splunk SOAR (On-premises) instance.
  
  SSH <username>@<primary_phantom_hostname>
2. SSH to your secondary and warm standby Splunk SOAR (On-premises) instance.
  
  SSH <username>@<warm_standby_phantom_hostname>
3. In both sessions, elevate to root.
  
  sudo su -
On the primary instance of Splunk SOAR (On-premises), perform the restore. See Restore Splunk SOAR (On-premises) from a backup.
On the primary instance of Splunk SOAR (On-premises), disable warm standby.

phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --primary-mode --off
On the secondary instance of Splunk SOAR (On-premises), disable warm standby.

phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --standby-mode --off
On the secondary instance of Splunk SOAR (On-premises), stop all Splunk SOAR (On-premises) services.

/<PHANTOM_HOME>/bin/stop_phantom.sh

Failing to stop these services on the secondary instance results in two active instances operating independently, polling for data and executing automated actions. This can result in data loss or other undesired results.
On the primary instance of Splunk SOAR (On-premises), configure it to be the primary instance for warm standby. You are prompted to give passwords for the Splunk SOAR (On-premises) user, the PostgreSQL database replication user, and the information for creating a self-signed SSL certificate.

phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --primary-mode --configure --primary-ip <primary_ip> --standby-ip <standby_ip>
On the secondary instance, configure it to be the warm standby instance.

phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --standby-mode --configure --primary-ip <primary_ip> --standby-ip <standby_ip>
On the both instances of Splunk SOAR (On-premises), verify that warm standby is replicating on each Splunk SOAR (On-premises) instance.

phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --status

Example output from Splunk SOAR (On-premises) primary:

========= Processed Params =========
  Instance looks like Primary
   DB replication configured with Standby set to:  <warm_standby_ip>/32
   DB replication currently streaming
   Vault sync configured
=========  Script Done =========

Example output from Splunk SOAR (On-premises) secondary or warm standby:

========= Processed Params =========
  Instance looks like Standby
   DB replication configured
   rsync configured
=========  Script Done =========

Restore a backup from a warm standby primary to a new Splunk SOAR (On-premises) instance

When you restore a backup of a Splunk SOAR (On-premises) warm standby primary to a new instance that you want to become the new primary, you must update the warm standby configuration and move several keys to the secondary instance.