Migrate DB Connect deployment to DB Connect 3

DB Connect 3 rewrote the JDBC backend and data collection configuration files. You must migrate the environment, input, output, and lookup configurations you created in DB Connect 2 to DB Connect 3 (version 3.0.0 and later).

Since the deployment scenarios vary among different users, the script may fail to migrate your configurations. Therefore, you may require to redefine your configurations after DB Connect 3 installation

Use the migration script dbx_app_migration.py located at $SPLUNK_HOME/etc/apps/splunk_app_db_connect/bin to migrate the following artifacts from DB Connect 2 to DB Connect 3:

• JDBC Drivers (If you use complex drivers such as Oracle, Spark SQL, or Teradata in your DB Connect 2 deployment, be sure to complete the extra step to manually migrate those drivers to DB Connect 3. Drivers that require many files and which have several dependencies do not automatically migrate.)
• JVM Options, such as JRE installation path, RPC server port, or memory configuration
• connections and identities
• database inputs
• database outputs
• database lookups

User permissions

Splunk recommends you to run the migration scripts using Splunk Python. Run the following script from a Linux Terminal or Windows Command Prompt :

$SPLUNK_HOME/bin/splunk cmd python $SPLUNK_HOME/etc/apps/splunk_app_db_connect/bin/dbx_app_migration.py

Use -h to see all options for the app_migration command.
To migrate the DB Connect install, the user must have full read and write access to $SPLUNK_HOME/var, $SPLUNK_HOME/etc/apps/splunk_app_db_connect, and their subdirectories.
Depending on how you installed your instance of Splunk software, you may need to execute migration as a user with higher privileges than usual, such as root or LocalSystem, in order to have full access to files and processes the Splunk Enterprise service owns. See Use access control to secure Splunk data for details.

Migration paths

The following matrix summarizes your upgrade options from the version(s) of DB Connect you're running currently and the migration path to DB Connect 3.

From	Action
DB Connect 2.3.0 DB Connect 2.3.1 DB Connect 2.4.0	Use the migration scripts during the upgrade process.
DB Connect 2.x.x version prior to 2.3.0	1. Upgrade to DB Connect 2.4.0 first, no migration needed for this step. See single server deployment on how to install DB Connect. 2. Migrate to DB Connect 3 using migration scripts.
DBX 1.x.x	For DB Connect 1.x.x installation with few configurations, use Migrating from DB Connect 1.x.x to DB Connect 2.x.x and then follow Migrate to DB Connect 3 using migration scripts. There are known limitations to the DB Connect 2.1.x migration scripts, and a more complex environment may require a reinstallation of DB Connect 3 migrate manually.

If you have difficulties or errors when migrating DB Connect to the latest version, contact Splunk Support for migration support.

Upgrade and migrate instructions

The upgrade and migration processes are different for a single instance and for a distributed deployment. Follow the instructions below to upgrade and migrate DB Connect.

• Migrate DB Connect 2 to DB Connect 3 on a single instance.
• Migrate DB Connect 2 to DB Connect 3 on distributed deployment.

Migrate DB Connect 2 to DB Connect 3 on a single instance

1. You can upgrade to DB Connect version 3 from version 2.3.0 or later by clicking the Upgrade button in the Apps listing, or by downloading the latest installation package from https://splunkbase.splunk.com/app/2686/ and install DB Connect.
2. Restart your Splunk instance.
3. Run the migration script dbx_app_migration.py located at $SPLUNK_HOME/etc/apps/splunk_app_db_connect/bin

   dbx_app_migration.py [-h] [-scheme SCHEME] [-port PORT] [-verbose]

   • -scheme: Optional. URI scheme of Splunk platform, either http or https, the default is https.
   • -port: Optional. Port number of Splunk platform, the default is 8089.
   • -verbose: Whether to enable verbose logging, the default is False.
   • -help or -h: Optional.
4. When prompted, enter the user name and password of the Splunk instance.
5. The migration scripts back up your existing drivers, connections, inputs, outputs, and lookups configuration in $SPLUNK_HOME$/etc/apps/splunk_app_db_connect/migration_backups and lists actions it performs
6. View the actions the migration scripts list and choose whether to perform the migration based on your situation. To proceed, respond to the message with Y.

   Apply above actions, continue?[Y/n]Y

   • If the migration succeeds, the status shows 100% done and the successful message appears.
   • If there are errors during the migration, the errors appear during the migration, for example,

   Performing action [3/29]: |█████---------------------------------------------| 10.3% Done!
   failed to execute action 3, cause:HTTP 400 Bad Request -- Object id=conn_mcafee_epo cannot be deleted in config=db_connections.
   Y to skip failed action and continue, n to abort the migration? [Y/n]
   

   You can decide whether to continue or abort the migration.

   • If you choose to ignore the error and proceed, type Y.
   • If you want to abort the migration, manually correct the error and migrate again. Copy the backup files under $SPLUNK_HOME$/etc/apps/splunk_app_db_connect/migration_backups/splunk_app_db_connect back to $SPLUNK_HOME$/etc/apps/splunk_app_db_connect before the next migration.
7. (Optional).When migrating from DB Connect 2 to DB Connect 3, you must move JDBC driver files to new locations. You cannot automatically migrate complex drivers that require many files and have many dependencies such as, Oracle, Spark SQL and Teradata. For those complex drivers, you must manually move the dependent JAR files in the $SPLUNK_HOME/etc/apps/splunk_app_db_connect/drivers/<JDBC driver name>-libs folder. See Install database drivers for more information.
8. Restart Splunk when the migration succeeds.
9. Once the migration scripts have completed, launch DB Connect 3. Click Datalab to view your transferred database inputs, outputs and lookups.

Note: If the task server cannot start after migration, you must kill the RPC server process manually and restart Splunk Enterprise again. This is a known issue when migrating on Ubuntu operating system.

Migrate DB Connect 2 to DB Connect 3 on a distributed deployment

Migrate DB Connect on the search head cluster

If you installed DB Connect on a search head cluster, use the cluster deployer to migrate from DB Connect 2 to DB Connect 3. If you are using scheduled inputs or outputs on the search head cluster, you must copy the configuration of a search head cluster node to a heavy forwarder, then migrate that forwarder to DB Connect 3. Because configuration replication between SHC nodes does not include the cluster deployer, you cannot migrate scheduled tasks to DB Connect 3 unless you copy the configuration filesfrom an SHC node to a forwarder.

1. Scheduled inputs and outputs cannot run on a search head cluster in DB Connect 3. If you have scheduled inputs and outputs on DB Connect 2, you must copy the files under $SPLUNK_HOME/etc/apps/splunk_app_db_connect to a heavy forwarder before migrating.
2. Copy the JDBC driver from one of the cluster node to the subdirectories under $SPLUNK_HOME/etc/shcluster/apps/splunk_app_db_connect/bin/lib on the deployer.
3. To upgrade DB Connect on the deployer, download the latest version of DB Connect from https://splunkbase.splunk.com/app/2686 and untar the installation package to $SPLUNK_HOME/etc/shcluster/apps/splunk_app_db_connect. Note that you cannot use Splunk web to perform this step.
4. Run the

   splunk apply shcluster-bundle -target <URI>:<management_port> -auth <username>:<password>

   command on the deployer to distribute the upgrade bundle to search head cluster members.
5. To migrate the configurations on the deployer, see Migrate to DB Connect 3 using the migration scripts. Note that migration scripts can detect that you are migrating on the deployer and ask you to provide the API endpoint and credentials of one cluster node.
6. Run the

   splunk apply shcluster-bundle -target <URI>:<management_port> -auth <username>:<password>

   command to distribute the migration to search head cluster members and restart Splunk Entperprise when the migration succeeds.

Note the following:

• The -target parameter specifies the URI and management port for any member of the cluster, for example, https://10.0.1.14:8089. You specify only one cluster member but the deployer pushes to all members. This parameter is required.
• The -auth parameter specifies credentials for the deployer instance.

Migrate DB Connect on heavy forwarder

If you have schedule inputs and outputs on the search head cluster, back up your existing inputs and outputs on the heavy forwarder before migration. The migration procedure for a heavy forwarder is the same as migration on a single instance. See migrate DB Connect 2 to DB Connect 3 on a single instance.

Migrate DB Connect resource pools

Because of the increased vertical scale of DB Connect 3, resource pooling is not available. The tasks that you perform on the resource pool master do not distribute to other resource pool members after the migration. If you are using resource pool in DB Connect 2, see performance expectations to redesign your deployment. You can remove the other resource pool members to conserve resources or configure them as separate heavy forwarders to scale your deployment.

• If you are running resource pool master and resource pool members all on heavy forwarders: The migration procedure is the same as on a single instance. See migrate DB Connect 2 to DB Connect 3 on single instance for more details.
• If you are running resource pool master on a search head cluster and resource pool members on heavy forwarders: Back up your configurations to one of the resource pool members first, then migrate this pool member. The migration procedure is the same as the migration on a single instance. See migrate DB Connect 2 to DB Connect 3 on a single instance for details.

If you have difficulties or errors during the migration, see troubleshooting or contact Splunk Support.