Use ibackup.pyc with warm standby

The warm standby and the backup and restore features require careful planning to use together.

Warm standby and ibackup features of use the Write Ahead Logging feature in PostgreSQL. When you configure a deployment to use both warm standby and ibackup, you must configure warm standby first. After restoring a deployment with ibackup, you must update the warm standby configuration.

Configuring warm standby after configuring ibackup archives all existing backups. Archiving all of the backups prevents new backups from being generated or existing backups from being used in a restore.

You can generate new backups once you run ibackup with the --setup option.

Restore a system configured for warm standby

In a warm standby configuration, when the primary instance is restored from a backup, you must update the warm standby configuration.

Prerequisites

You need the following information to update your warm standby configuration:

• Password for the user on the secondary instance. If the user does not have a password, you must set one.
• Password for the PostgreSQL database replication user.
• Configuration information for creating the SSL certificate:
  • Country code
  • State code
  • Organization
  • Organization unit
  • Domain
  • Email

Restore a backup from a warm standby primary to the same instance

When you restore a backup of a warm standby primary to the same instance, the warm standby configuration must be updated.

To update the warm standby configuration, perform the following steps:

1. Open a terminal session for both the primary and secondary instances. Keep these sessions open until you complete these steps.
   1. From the command line, SSH to your primary instance.
      
      SSH <username>@<primary_phantom_hostname>
   2. SSH to your secondary and warm standby instance.
      
      SSH <username>@<warm_standby_phantom_hostname>
   3. In both sessions, elevate to root.
      
      sudo su -
2. On the primary instance of , perform the restore. See Restore from a backup.
3. On the primary instance of , disable warm standby.
   
   phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --primary-mode --off
4. On the secondary instance of , disable warm standby.
   
   phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --standby-mode --off
5. On the secondary instance of , stop all services.
   
   /<PHANTOM_HOME>/bin/stop_phantom.sh
   
   

   Failing to stop these services on the secondary instance results in two active instances operating independently, polling for data and executing automated actions. This can result in data loss or other undesired results.

6. On the primary instance of , configure it to be the primary instance for warm standby. You are prompted to give passwords for the user, the PostgreSQL database replication user, and the information for creating a self-signed SSL certificate.
   
   phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --primary-mode --configure --primary-ip <primary_ip> --standby-ip <standby_ip>
7. On the secondary instance, configure it to be the warm standby instance.
   
   phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --standby-mode --configure --primary-ip <primary_ip> --standby-ip <standby_ip>
8. On the both instances of , verify that warm standby is replicating on each instance.
   
   phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --status

Example output from primary:

========= Processed Params =========
  Instance looks like Primary
   DB replication configured with Standby set to:  <warm_standby_ip>/32
   DB replication currently streaming
   Vault sync configured
=========  Script Done =========

Example output from secondary or warm standby:

========= Processed Params =========
  Instance looks like Standby
   DB replication configured
   rsync configured
=========  Script Done =========

Restore a backup from a warm standby primary to a new instance

When you restore a backup of a warm standby primary to a new instance that you want to become the new primary, you must update the warm standby configuration and move several keys to the secondary instance.

To update the warm standby configuration, perform the following steps:

1. Open a terminal session for both the primary and secondary instances. Keep these sessions open until you complete these steps.
   1. From the command line, SSH to your primary instance.
      
      SSH <username>@<primary_phantom_hostname>
   2. SSH to your secondary and warm standby instance.
      
      SSH <username>@<warm_standby_phantom_hostname>
   3. In both sessions, elevate to root.
      
      sudo su -
2. On the primary instance of , perform the restore. See Restore from a backup.
3. On the primary instance of , disable warm standby.
   
   phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --primary-mode --off
4. On the secondary instance of , disable warm standby.
   
   phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --standby-mode --off
5. On the secondary instance of , stop all services.
   
   /<PHANTOM_HOME>/bin/stop_phantom.sh
   
   

   Failing to stop these services on the secondary instance results in two active instances operating independently, polling for data and executing automated actions. This can result in data loss or other undesired results.

6. Copy these files from the new primary instance of to the secondary:
   1. /<PHANTOM_HOME>/keystore/private_key.pem
   2. /<PHANTOM_HOME>/www/phantom_ui/secret_key.py
7. On the secondary instance of , set the permissions, ownership, and SELinux security contexts for the files you copied to the secondary.
   1. chmod 0640 /<PHANTOM_HOME>/keystore/private_key.pem /<PHANTOM_HOME>/phantom/www/phantom_ui/secret_key.py
   2. chown root:phantom /<PHANTOM_HOME>/keystore/private_key.pem
   3. chown phantom:phantom /<PHANTOM_HOME>/www/phantom_ui/secret_key.py
   4. restorecon /<PHANTOM_HOME>/keystore/private_key.pem /<PHANTOM_HOME>/www/phantom_ui/secret_key.py
8. On the primary instance of , configure it to be the primary for warm standby. You are prompted to give passwords for the user, the PostgreSQL database replication user, and the information for creating a self-signed SSL certificate.
   
   phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --primary-mode --configure --primary-ip <primary_ip> --standby-ip <standby_ip>
9. On the secondary instance, configure it to be the warm standby instance.
   
   phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --standby-mode --configure --primary-ip <primary_ip> --standby-ip <standby_ip>
10. On both instances of , verify that the warm standby instance is replicating on each instance.
    
    phenv python /<PHANTOM_HOME>/bin/setup_warm_standby.pyc --status

Example output from primary:

========= Processed Params =========
  Instance looks like Primary
   DB replication configured with Standby set to:  <warm_standby_ip>/32
   DB replication currently streaming
   Vault sync configured
=========  Script Done =========

Example output from secondary or warm standby:

========= Processed Params =========
  Instance looks like Standby
   DB replication configured
   rsync configured
=========  Script Done =========