Splunk® Enterprise

Managing Indexers and Clusters of Indexers

Download manual as PDF

Download topic as PDF

Migrate non-clustered indexers to a clustered environment

Read the topic "Key differences between clustered and non-clustered deployments of indexers" carefully if you plan to migrate your current set of indexers to an indexer cluster. That topic describes issues that you must understand before initiating the migration process.

You can add a non-clustered indexer to a cluster (as a peer node) at any time. To do so, just enable the indexer as a peer, as described in "Enable the peer nodes".

Once the indexer has been made a peer, it participates in the cluster the same as any other peer. Any new data coming into the peer gets replicated according to the cluster's replication factor, and the peer is also a candidate for receiving replicated data from other peers. Data already on the indexer does not get automatically replicated, but it does participate in searches, as described below.

Important: Before migrating an indexer from non-clustered to clustered, be certain of your needs. The process goes in one direction only. There is no supported procedure for converting an indexer from clustered to non-clustered.

Manage legacy data

The term "legacy data" means data stored on an indexer prior to its conversion to a cluster peer.

How the cluster handles legacy indexed data

When you add an existing indexer to the cluster, the cluster does not replicate any buckets that are already on the indexer.

Buckets already on the indexer prior to its being added to the cluster are called "standalone" buckets. Searches will still occur across those buckets and will be combined with search results from the cluster's replicated buckets.

Is there any way to migrate my legacy data?

Because of the high processing cost of converting standalone buckets to replicated buckets (due to the need to make multiple searchable and non-searchable copies of those buckets to fulfill the cluster's replication and search factors), it is generally a bad idea to attempt to do so, particularly in the case of indexers with large numbers of standalone buckets. There is no supported procedure for this conversion. If, however, you have a strong need for such a conversion, contact Splunk Professional Services to discuss the trade-offs and requirements for this operation.

Migrate apps to a cluster

In your current, non-clustered environment, you might be using deployment server to distribute apps to your set of indexers. You will no longer be able to do so once you convert the indexers to cluster peer nodes. See "Key differences between clustered and non-clustered deployments of indexers" for details.

To distribute apps to peer nodes, you must instead use the configuration bundle method described in "Update common peer configurations and apps". You place the apps in a special location on the master node, and the master pushes the apps to each indexer when you first enable it as a peer. If you need to add or change apps later on, you make the changes and additions on the master and then tell the master to push the updated configuration bundle to the entire set of peers.

For information on how the configuration bundle method compares with the deployment server, see "Cluster peer management compared to deployment server".

Important: You must use the configuration bundle method to distribute apps to peer nodes; the deployment server is unsupported for that purpose.

How to migrate apps

It's recommended that you migrate apps at the time that you enable the cluster. The procedure described below describes how to do that.

Important: You must be familiar with the preceding topics in the current chapter before attempting this procedure. Start with "Indexer cluster deployment overview" and read onwards until you return to this topic.

This procedure assumes that you are starting with a distributed search environment with a set of indexers configured as deployment clients of a deployment server. The deployment server is being used to push apps to the indexers. It is these indexers that you will be converting to cluster peer nodes. Once they've become peer nodes, you can no longer use the deployment server to push apps to them; instead, you must use the configuration bundle method.

To migrate your apps at the time of cluster enablement, follow these steps:

1. Enable the master, as described in "Enable the master node".

2. On the deployment server, locate the set of apps that it pushes to the indexers that you're planning to migrate. These should be under the deployment server's $SPLUNK_HOME/etc/deployment-apps directory.

3. Copy these apps from the deployment server to the cluster master's $SPLUNK_HOME/etc/master-apps directory. See "Structure of the configuration bundle" for information on the master-apps directory.

4. Examine each app in master-apps for any indexes.conf file(s). In those files, locate any stanzas that define new indexes. In each of those stanzas, add this attribute/value pair:

repFactor=auto

This enables replication for the index. See "The indexes.conf repFactor attribute" for more information.

Note: If you are the creator and maintainer of the app, you can also make this change in $SPLUNK_HOME/etc/master-apps/<appname>/default/indexes.conf.

5. Convert each indexer to a cluster peer, one at a time:

a. Reconfigure the indexer so that it's no longer a deployment client.
b. Delete the deployed apps from the indexer's $SPLUNK_HOME/etc/apps directory. Leave the default apps, such as Search, in place.
c. Enable the indexer as a cluster peer, as described in "Enable the peer nodes".
d. Restart the peer to complete the enablement process.
e. Verify that the master has pushed the expected set of apps to the peer's $SPLUNK_HOME/etc/slave-apps directory.

6. Once all peers have been enabled, go to the master dashboard and verify that the expected set of indexes is being replicated. This dashboard is described in "View the master dashboard".

For more information on configuring cluster peers, see these topics:

PREVIOUS
Use indexer clusters to scale indexing
  NEXT
Upgrade an indexer cluster

This documentation applies to the following versions of Splunk® Enterprise: 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11, 6.5.0, 6.5.1, 6.5.1612 (Splunk Cloud only), 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.5.10, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.6.8, 6.6.9, 6.6.10, 6.6.11, 6.6.12, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 8.0.0, 8.0.1


Comments

Cdoebert - Thanks for pointing out that error. The step has been updated.

Sgoodman, Splunker
June 30, 2017

5b. Delete the apps in the indexer's $SPLUNK_HOME/etc/apps directory.

This trashed the first indexer I did it on. Does this mean to say "delete the *deployed* apps"?

Cdoebert
June 26, 2017

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters