2.0.x to 2.1.x Migration Instructions

2.0.x to 2.1.x Migration Instructions

Migration is the process of converting a Splunk 2.0 index to Splunk 2.1 format.

Migration is a non-reversible operation. The two versions use incompatible storage schema on disk.

Why a migration process?

This is the last time you will have to schedule a migration to upgrade Splunk.

Normally Splunk installations are compatible across release versions. But Splunk 2.1 introduces new search and index features such as delete:: as well as much higher performance. This required us to implement a different data schema on disk. Splunk 2.1 is much more space efficient than Splunk 2.0 - the overall amount of space taken by Splunk Server will usually be less after migration, and the server will run faster as well. To allow these new features to work with 2.0 and earlier data, the Splunk 2.1 server must first migrate 2.0 indexes to the 2.1 schema.

Beginning with Splunk 2.2, indexed data from previous versions of Splunk will continue to be searchable without a migration process. New features may not be available on events indexed with previous versions of Splunk, but no previous functionality on those events will be lost.

How long does it take?

Migrating a 100 million event index takes about 4 hours on typical hardware.

Will I lose my data?

During the migration process, the Splunk Server cannot be searched and any incoming events not cached to file before the Splunk Server indexes them will be permanently lost. This includes syslog port 514, Splunk-2-Splunk connections, and any other events accessed through TCP, UDP or FIFO.

Splunk-2-Splunk connections are not compatible between 2.0 and 2.1 servers. The connection will be down from the time either end is shut down until both ends are back up, and the receiving end has completed migrating its index from 2.0 to 2.1 format.

Avoid data loss

Please review and follow these steps to prevent the Splunk 2.0 to 2.1 migration process from resulting in lost data.

Make a copy first

The most obvious way to avoid data loss is to make a copy of your Splunk 2.0 installation, migrate one copy, and keep the other as a backup. That way, if the non-reversible migration process is interrupted or fails for any reason, you will still have a usable copy of your Splunk 2.0 indexes.

Don't migrate > 100 million events

If your index contains more than 100 million events, the migration program will not work for you. Set up a new, parallel 2.1 server instead of attempting to migrate your 2.0 index to 2.1. Yes, this will require you to search old and new events separately, but it will eliminate the risk of unacceptable downtime.

Splunk-2-Splunk setups

To upgrade Splunk-2-Splunk multi-server configurations from 2.0 to 2.1:

1. Shut down your forwarding server(s) first. Be sure their splunkd processes are stopped. This will prevent them from transmitting data to the receiving server via TCP while it is down.
2. Update the receiving server to 2.1.
3. Restart the receiving server. It will prompt you to migrate its indexes when it first comes up after the upgrade. See the note above on estimated migration time.
4. After the receiving server has migrated its index from 2.0 to 2.1 and returned to normal operation, update and restart each of your forwarding servers.
5. As each forwarding server restarts running Splunk 2.1, it will forward its backlog of event to the 2.1 receiving server.

UDP, TCP, and FIFO inputs

If your Splunk Server receives syslog or other events via UDP or TCP connections rather than indexing them from local files, or if it reads from one or more FIFO queues:

1. Set up an alternate temporary recording mechanism, such as a parallel syslog deamon, to write these events to file.
2. Shut down the Splunk Server and upgrade it from 2.0 to 2.1.
3. After the Splunk Server has migrated its index from 2.0 to 2.1 schema and returned to normal operation, redirect your non-cached inputs to the Splunk Server again.
4. Index any archive files created during the migration process by your alternate service.

Instructions (native package)

1. After backing up the 2.0.X installation, transfer the native package to the

server you wish to migrate.

1. Stop splunk manually:

# $SPLUNK_HOME/bin/splunk stop

The native package will attempt to stop the server for you, but known issues with

the 2.0.X software may prevent it from shutting down. If the 2.0.X

Splunk Server isn't shutting down after being given the stop command, kill

the splunkd processes with kill -9.

1. Using the appropriate command install the 2.1 Splunk Server software over

the 2.0.X installation. The prefix supplied to the native package should be

the absolute path to the splunk directory. For example, if Splunk Server is

installed in /usr/local/splunk then the prefix supplied to the native package

installer should be /usr/local. The default is /opt .

1. Once the native package utility has completed is operation change into

the splunk directory and start the server:

# ./bin/splunk start

a. Splunk will prompt you to confirm the migration process. Answer "y".

b. Splunk will display the Splunk Server 2.1 license agreement. Page through it

and accept the license.

c. Splunk Server 2.1 will first migrate your 2.0.X configuration. It will then migrate

your indexed data in two phases. During this time it will display progress messages.

Migration is not complete and the Splunk Server is not available until the

command line prompt returns.

During migration you may see output that indicates an error, or that files

aren't found. More often than not it only means you didn't have that aspect

of 2.0.X to migrate. Please give the migration process an opportunity to complete and

check your installation before contacting support.

1. Once the splunk start command has completed and your shell prompt has

returned, the migration process is complete. Login to the Splunk Server's web interface.

Spot check your data input configuration and perform some searches to ensure

system responsiveness. If it works as expected, you're done!

Instructions (tar file)

1. After backing up the 2.0.X installation transfer the tar archive to the

server you wish to migrate.

1. Stop splunk manually:

# $SPLUNK_HOME/bin/splunk stop

There are known issues with the 2.0.X software that may prevent it from shutting down. If

the 2.0.X Splunk Server isn't shutting down after being given the stop

command, kill the splunkd processes with kill -9.

1. Create a backup copy of your configuration that Splunk Server will

use as the basis for migration. Change to your splunk directory and make a backup

copy of your confiuration files:

# mv etc etc.bak

Don't deviate from this exact command. When the Splunk Server starts, the old etc directory needs to be gone, and the server will only look for an etc.bak from which to migrate old configuration files. Any change to the above command risks making the server inoperable.

1. Place the tar archive next the existing 2.0.X splunk directory and

extract it. For example, if your splunk directory is /opt/splunk then

you should place the tar archive in /opt and run this command:

# tar xvzf splunk*.tgz

If your tar command doesn't have integrated support for compressed files,

you will need to decompress it first.

1. Change into the splunk directory and follow the instructions in the file README_TAR_INSTALL.

# cd /opt/splunk

# cat README_TAR_INSTALL

1. Start the Splunk Server:

# ./bin/splunk start

a. The Splunk Server will prompt you to confirm the migration process. Answer "y".

b. Splunk will display the Splunk Server 2.1 license agreement. Page through it and accept the license.

c. Splunk Server 2.1 will first migrate your 2.0.X configuration. It will then migrate

your indexed data in two phases. During this time it will display progress messages.

Migration is not complete and the Splunk Server is not available until the

command line prompt returns.

During migration you may see output that indicates an error, or that files

aren't found. More often than not it only means you didn't have that aspect

of 2.0.X to migrate. Please give the migration process an opportunity to complete and

check your installation before contacting support.

1. Once the splunk start command has completed and your shell prompt has

returned, the migration process is complete. Login to the Splunk Server's web interface.

Spot check your data input configuration and perform some searches to ensure

system responsiveness. If it works as expected, you're done!

Parallel 2.0 / 2.1 setup

1. Install Splunk 2.1 in a different directory from Splunk 2.0.
2. Start the 2.1 server. If the Splunk 2.0 server is already running on ports 8000 / 8001 / 8089, the 2.1 server will prompt for different ports.
3. Export the 2.0 server's event type tags and sourcetype renames, if there are any, to a temporary file by running these commands on the 2.0 server. Be sure to use the 2.0 server's .bin/splunk command.

# splunk export -s > /tmp/sourcetypes

# splunk export -t > /tmp/tags

1. Import the event type tags and sourcetype renames by running these commands on the 2.1 server. Be sure to use the 2.1 server's .bin/splunk command.

# splunk import -s < /tmp/sourcetypes

# splunk import -t < /tmp/tags

1. Run the following command on the Splunk 2.1 instance to migrate your environment from your 2.0 installation to your 2.1 install.

# ./splunk migrate env /path/to/2.0/splunk_home

This command will migrate your Saved Splunks, Live Splunks, data input configurations, processing properties and regex definitions. It will not migrate your 2.0.x indexed data. To migrate your indexed data, you will need to do an overlay install as described above.

Help and support

Error messages

We've collected a list of known Installation Errors and their fixes. If you encounter any errors or warnings during the process - or anything that seems wrong - please check there first.

More help

After migration, if your Splunk Server is not functional or some of your data appears to be missing,

please contact Splunk Support with the following information:

1. The OS version and Splunk version you were attempting to migrate from
2. A copy of the CLI output from the splunk start command
3. A copy of splunkd.log from $SPLUNK_HOME/var/log/splunk/splunkd.log
4. A copy of $SPLUNK_HOME/etc/myinstall/pluginConfs/multiIndexer.xml

This information will enable Splunk to respond to your support request

in a timely and efficient manner. Call your support representative or

email support@splunk.com.

• Was this topic useful?
• Post a comment